In Part 1, we introduced the M×N scaling strategy to test the performance limits of our Time Series Database (TSDB). In Part 2, we shared how our team’s feedback led us to drop a complex stream processing design in favor of a simpler, more practical approach using our existing data writer service.
Now comes the moment of truth: actually testing the TSDB’s limits and finding the real bottlenecks.
What we found changed how we look at system performance. The bottlenecks were not where we expected them to be. Fixing them required us to look far beyond our application code. This is the story of how we uncovered our database’s true limits and learned to scale beyond them.
Our goal was simple: push the TSDB to its breaking point. Using our M×N strategy, we planned to turn up the data volume step by step until the system buckled.
Here is what we expected to see in theory:
When we ran the tests, the results were sobering:
| Test | Parallelism | M | Expected/min | Actual/min | Efficiency |
|---|---|---|---|---|---|
| 1 | 4 | 6 | 130.2M | ~73M | 56% |
| 2 | 6 | 6 | 130.2M | ~104M | 80% |
| 3 | 6 | 12 | 260.4M | ~130M | 50% |
| 4 | 6 | 20 | 434M | ~130M | 30% |
The hard truth: When we pushed more data (increased M), our efficiency dropped fast. We hit a brick wall at about 140-145 million metrics per minute, no matter how we tweaked the application settings.
We checked the TSDB monitoring dashboard, and to our surprise, the database was barely breaking a sweat:
The realization: The database wasn’t the problem. Our own application was too slow to generate enough load to stress the database!
Before we could test the TSDB’s limits, we had to fix our own code.
The Issue: When our application sent data to the database, it stopped and waited for a response. During this waiting time, the thread was blocked and couldn’t process any new messages from Kafka.
Before: Synchronous Architecture
<div class="mermaid">
graph TD
subgraph "Consumer Thread"
A[Kafka Poll] --> B[Data Amplification]
B --> C[timeSeriesService.write()]
C -- "I/O Wait (seconds)" --> D[Write Complete]
D --> A
end
</div>
After: Asynchronous Architecture with Virtual Threads
<div class="mermaid">
graph TD
subgraph "Consumer Thread"
A[Kafka Poll] --> B[Data Amplification]
B --> Q[Submit Write Task]
Q --> A
end
subgraph "Virtual Thread Pool"
Q --> VT1[Virtual Thread 1: Write Batch 1]
Q --> VT2[Virtual Thread 2: Write Batch 2]
Q --> VTx[Virtual Thread N...]
end
</div>
The Fix: We used Java 21’s Virtual Threads to make the database writes asynchronous. Now, the main thread simply hands the write task to a virtual thread and immediately goes back to fetching more data.
The Issue: Even after fixing the I/O problem, our CPU usage was still too high. Why? Because we were creating millions of temporary HashMap objects every minute during the M×N data multiplication. This caused the Garbage Collector (GC) to work overtime, slowing down the whole system.
The Fix: Instead of creating a new HashMap in every loop, we pre-created the required maps once and reused them. This cut our object creation down massively.
Mission Accomplished: With our application running smoothly, it was time to find the database’s real limits.
Now we could push harder. We cranked M up to 12, aiming for over 260M metrics per minute.
The Surprise: Performance didn’t go up. It actually dropped to around 100M/minute. We also started seeing Slow write request warnings that lasted 7 to 9 seconds!
We had finally found the TSDB’s breaking point. But why was it breaking?
We dug back into the TSDB dashboards:
This wasn’t a bug; it was a built-in safety feature. Our TSDB (VictoriaMetrics) uses a component called vminsert which protects itself from overload by limiting concurrent processing to CPU cores × 2.
The Math:
The Reality: Our hundreds of parallel write requests were piling up in the database’s waiting line. They had to wait for one of those 16 slots to open up. That waiting time is exactly what caused the 7-9 second delays.
This is exactly what we wanted to find. We discovered a hard, physical limit that would affect our production systems.
What we learned:
Now we knew the problem: the 16-slot concurrency limit. Since this limit is tied to CPU cores, the only way to get more slots was to get more cores.
Instead of trying to hack the database settings, we chose a straightforward infrastructure upgrade:
The Solution: High-Performance Physical Servers
The Migration Impact:
# Before: Virtual Machines (VMs)
Servers: 8 × VMs (8 cores, 32GB)
Concurrency Limit: 8 × 2 = 16 slots per server
Capacity: ~120M metrics/min
# After: Physical Machines (PMs)
Servers: 8 × PMs (40 cores, 256GB)
Concurrency Limit: 40 × 2 = 80 slots per server
Expected Jump: A massive 5x increase in capacity!
This decision highlights some great lessons about real-world engineering:
This upgrade instantly gave us 5x more capacity and set us up perfectly for our final tests.
Choosing the “Direct Writer” approach back in Part 2 proved to be a brilliant move:
The journey from the M×N strategy concept to successfully testing our TSDB’s limits was challenging but incredibly rewarding. Through application-level optimizations and strategic infrastructure scaling, we achieved our ultimate goal.
Mission Accomplished: From 60 Seconds to 1 Second We successfully completed the PoC, proving our system can seamlessly handle the transition from a 60-second collection interval down to a 1-second interval (M=60). By upgrading our infrastructure to 40-core Physical Machines, we broke through the initial concurrency bottlenecks and confidently absorbed the massive influx of over 400M+ metrics per minute.
Validating Our TSDB Choice (VictoriaMetrics)
Perhaps the most important outcome of this extreme load testing was the validation of our core architectural decision. Hitting the vminsert concurrency limit early on wasn’t a failure of the database; rather, it was a testament to its self-protective design. Once provided with the appropriate hardware, VictoriaMetrics demonstrated incredible vertical scalability and raw ingestion efficiency.
It proved beyond a doubt that it can handle the punishing load of 1-second granularity metrics across our entire cloud infrastructure without breaking a sweat. This PoC gave us the definitive answer we needed: our choice of VictoriaMetrics as our TSDB was absolutely the right one.
What’s Next? With our ingestion capabilities proven for 1-second intervals, we’re now ready to tackle the next challenge: optimizing for cardinality explosion and complex query patterns on the read side. But that’s a story for another day.
This concludes our three-part series on scaling time series data processing. From initial strategy through team collaboration to discovering real-world database constraints, we’ve covered the complete journey from theoretical concept to practical system success.
]]>In Part 1, I explained the “M×N” strategy for scaling time series data. My original idea was to build a dedicated stream processing layer using Flink or Kafka Streams. It was a clean, theoretically perfect design that kept data transformation completely separate from data ingestion.
However, a “theoretically perfect” design in a whiteboard session often clashes with the gritty reality of a production environment.
Before diving into the Proof of Concept (PoC) development, we conducted a rigorous architectural review. This review process proved to be incredibly valuable, leading us to completely scrap the Flink idea in favor of a brutally simple, highly pragmatic alternative.
This post details that architectural pivot. It highlights how critically evaluating operational constraints guided us away from a complex new system and toward an efficient, battle-tested solution.
During our design review, we stress-tested the Flink proposal against the realities of our infrastructure. As we analyzed the operational requirements, we quickly identified two critical risks:
data writer service uses a custom parallel consumer specifically built to handle the severe network delays between our isolated data centers (IDCs). Standard Kafka consumers struggle in this environment. If we deployed Flink in a separate IDC, it would immediately hit this exact network bottleneck, invalidating our performance tests.It became clear that introducing a new stream processing engine would force us to solve infrastructure and network problems before we could even test the Time Series Database (TSDB).
We needed a new direction.
To solve these constraints, we pivoted to an elegant and highly practical alternative: Why not embed the M×N transformation logic directly into our existing data writer service and scale that horizontally?
The concept was simple: Have each instance of the data writer service read the same raw data from Kafka, but configure each instance to generate a specific, non-overlapping slice of the final M×N data.
Each writer instance handles both the M (time division) and N (cardinality expansion) dimensions internally. For example, one instance handles timestamps at 00s, 10s, and 20s. Another handles 30s, 40s, and 50s. Both instances also multiply the data to increase the cardinality (N) before writing it to the database.
// Inside the existing Data Writer Service
public class DataProcessor {
private final List<Integer> assignedTimestampOffsets; // M: time division
private final int cardinalityMultiplier; // N: data volume expansion
public void process(Metric metric) {
LocalDateTime baseTime = metric.getTimestamp().truncatedTo(ChronoUnit.MINUTES);
// M: Time division - split into multiple timestamps
for (int offset : assignedTimestampOffsets) {
// N: Cardinality expansion - increase data volume
for (int instanceOffset = 0; instanceOffset < cardinalityMultiplier; instanceOffset++) {
Metric transformed = metric.copy();
transformed.setTimestamp(baseTime.plusSeconds(offset));
transformed.setInstanceNo(metric.getInstanceNo() + instanceOffset);
emitToWriterQueue(transformed);
}
}
}
}
// Writer Instance A: handles 00s, 10s, 20s with 6x cardinality
DataProcessor writerA = new DataProcessor(
Arrays.asList(0, 10, 20), // M: 3 timestamps
6 // N: 6x data volume
);
// Writer Instance B: handles 30s, 40s, 50s with 6x cardinality
DataProcessor writerB = new DataProcessor(
Arrays.asList(30, 40, 50), // M: 6x data volume
6 // N: 6x data volume
);
Let’s do the math:
This approach achieved the exact same M×N scaling effect (36x data volume) as the complex Flink architecture, but completely bypassed the need for new infrastructure.
Most importantly, it leveraged a battle-tested service that had already solved our cross-IDC network latency issues.
We formalized the comparison to ensure we were making the right long-term trade-offs.
| Aspect | Flink/Stream Processor Approach | Direct Writer Scaling Approach | Analysis |
|---|---|---|---|
| Time to Market (PoC) | Slow | Fast | No new infrastructure or deployment pipelines needed. |
| Operational Risk | High | Low | Introduces an unproven component vs. scaling a stable, known one. |
| Resource Efficiency | Low | High | Avoids duplicating data in a second Kafka topic, saving massive storage. |
| M×N Implementation | High | High | Both approaches successfully implement the scaling strategy. |
| Architectural Purity | High | Low | Concerns are cleanly separated vs. writer handling both transformation and I/O. |
| Network Resilience | Low | High | Flink would struggle with our IDC latency; the writer already handles it perfectly. |
For our PoC, the decision was obvious. We happily traded textbook architectural “purity” for a massive gain in speed, resource efficiency, and network resilience.
This pivot from a theoretical ideal to a highly pragmatic solution reinforced several core principles of senior-level system design.
A “good” architecture is one that solves the business problem efficiently. Our goal was to test TSDB limits safely and quickly. By recognizing that building a perfect stream processing pipeline was a distraction from our actual goal, we saved weeks of engineering time.
You cannot design software in a vacuum. The specific network latency between our data centers was a hard constraint that immediately invalidated our theoretical Flink design. Truly robust architectures are built around—and optimized for—the unique limitations of their physical environments.
Every new component (like Flink) introduces operational overhead, deployment risk, and maintenance burdens. By strategically reusing and scaling an existing, battle-tested component, we achieved our high-throughput goal while keeping the system architecture as lean as possible.
This experience proved that the best architectures are rarely the most complex ones. They are the ones that balance technical rigor with operational reality to deliver results efficiently.
All technical content in this article is based on actual production experience. Specific system names and configuration values have been generalized for security.
]]>When running time series data processing systems, we often need to scale data ingestion. Two common scenarios are:
Initially, I took a simple approach: why not just copy the existing data multiple times? However, this “simple” method caused unexpected problems. It forced me to rethink the challenge from a completely different angle.
This article shares my trial-and-error process and the improved architecture we built as a result.
Current: Low-density metric data (e.g., N-minute intervals)
Target: High-density metric data (e.g., M-second intervals)
When faced with this, my first thought was to simply “copy the existing data by the required multiplier.”
// Initial implementation: Simple multiplication
public void processData(List<Metric> metrics) {
List<Metric> expandedMetrics = new ArrayList<>();
for (Metric metric : metrics) {
// Generate data by the target multiplier
for (int multiplier = 0; multiplier < TARGET_MULTIPLIER; multiplier++) {
Metric duplicated = metric.copy();
duplicated.adjustTimestamp(multiplier);
expandedMetrics.add(duplicated);
}
}
// Send all data to the sink at once
for (Metric expanded : expandedMetrics) {
sink.write(expanded); // OOM occurs due to the massive data volume!
}
}
This simple code quickly revealed serious issues:
1. OutOfMemoryError (OOM)
Existing data volume: X
Multiplier: N
Result: X * N items loaded into memory simultaneously → OOM
2. Garbage Collection (GC) Pressure
3. Scalability Limitations
These problems led me to ask a fundamental question: “Is this approach truly practical?”
Why did the first approach fail? Here are the main reasons:
The key insight: “The code wasn’t the problem. The way we thought about the data was the problem.”
Looking closer, our initial requirement actually hid two different test objectives:
Objective: More granular monitoring.
Method: 1-minute intervals → 1-second intervals (temporal refinement).
Impact: More data points for the *same* time series.
Test Target: TSDB write throughput and storage capacity limits.
Objective: System performance testing (measuring TSDB limits).
Method: Diversifying unique identifiers (e.g., server IDs, instance IDs).
Impact: An N-fold increase in the number of unique time series, i.e., N-fold cardinality.
Test Target: TSDB performance for indexing, metadata processing, and label-based queries.
Cardinality = The number of unique time series (a combination of metric name + labels).
※ The timestamp does not affect cardinality.
※ Different unique identifiers are treated as distinct time series.
I failed initially because I didn’t understand this distinction. I only focused on “increasing data volume.”
This realization demanded a new approach. I needed to drop the “load everything into memory” mindset. Instead, I needed to embrace stream processing while handling both the M and N dimensions.
I decided to tackle the more complex M-dimension (increasing data density) first. I looked at two alternative approaches.
Note: The N-dimension (increasing cardinality) is easy to implement. We simply add unique identifiers to the data inside each processing cluster. Therefore, we will focus on the harder M-dimension here.
My first idea was to distribute the data perfectly over time.
// Pseudocode: Memory Holding Method
public class WindowedProcessor {
private Map<String, List<Metric>> oneMinuteBuffer;
public void process(Metric metric) {
// Collect data for a 1-minute window
buffer.add(metric);
if (windowComplete()) {
// Dispatch at precise intervals (e.g., every 1 second)
sendAt("14:00:00", createMetrics(buffer, 0));
sendAt("14:00:01", createMetrics(buffer, 1));
sendAt("14:00:02", createMetrics(buffer, 2));
// ... (interval is configurable)
}
}
}
Advantages:
Disadvantages:
// Pseudocode: Immediate Transformation Method
public class ImmediateProcessor {
public void process(Metric metric) {
LocalDateTime baseTime = metric.getTimestamp()
.truncatedTo(ChronoUnit.MINUTES);
// Immediately transform one metric into M metrics upon arrival
for (int offset = 0; offset < INTERVAL_SECONDS; offset += TARGET_INTERVAL) {
Metric transformed = metric.copy();
transformed.setTimestamp(baseTime.plusSeconds(offset));
emit(transformed);
}
}
}
Advantages:
Disadvantages:
While debating between the two approaches, I had an “Aha!” moment.
I thought about how data actually arrives in a real monitoring environment.
Data flow in high-density monitoring:
Actual Arrival Time Metric Data Content
──────────────────────────────────────────
12:00:03 server1_cpu (timestamp: 12:00:00)
12:00:07 server2_cpu (timestamp: 12:00:00)
12:00:13 server1_cpu (timestamp: 12:00:10)
12:00:15 server3_cpu (timestamp: 12:00:00)
12:00:17 server2_cpu (timestamp: 12:00:10)
12:00:23 server1_cpu (timestamp: 12:00:20)
...
→ Characteristic: Each server sends data at its own, uncoordinated time.
→ Result: Data arrives as a continuous, irregular stream.
The Key Insight:
“In a real streaming environment, data arrives individually and continuously. Is there any good reason to buffer it and process it as a batch?”
This changed everything. In the real world:
Even though the Immediate Transformation method created small bursts, it actually mirrored real-world streaming much better than the buffering method!
Let’s look closely at the memory usage of the Immediate Transformation method.
public void process(Metric input) {
// M objects are created momentarily
for (int i = 0; i < M; i++) {
Metric transformed = input.copy(); // M objects
emit(transformed);
}
}
The conclusion? It uses O(M) memory. Yes, it creates M objects, but it emits them immediately. The garbage collector can clean them up right away. This is vastly more efficient than holding onto O(M × time_window) data in memory.
Someone asked me, “If we increase M, does that also increase cardinality?”
The answer is no: Changing the timestamp interval does not increase cardinality.
Cardinality only depends on the metric name + labels.
The timestamp does not change cardinality.
Example:
- cpu_usage{server="web01"} → 1 time series
- If we collect this every 10 seconds instead of 1 minute, it is still only 1 time series.
However, increasing M definitely increases the number of data points, which affects our storage and raw write performance.
In the end, I chose the Immediate Transformation method.
How we scaled the N-Dimension: We ran processing engines in parallel.
metric{server="web01"} became metric{server="web01", metric_id="1"}, metric{server="web01", metric_id="2"}→ Result: We generated N times the number of unique time series simply by changing the identifiers.
I built a two-dimensional strategy to control data volume and cardinality independently.
Why test both dimensions?
We must separate them because a TSDB handles raw data points very differently than it handles unique time series indexes.
// M-Implementation: Timestamp division
LocalDateTime baseTime = metric.getTimestamp().truncatedTo(ChronoUnit.MINUTES);
for (int offset = 0; offset < INTERVAL_SECONDS; offset += TARGET_INTERVAL) {
Metric transformed = metric.copy();
transformed.setTimestamp(baseTime.plusSeconds(offset));
emit(transformed); // Generates M times the data points
}
Effect:
// N-Implementation: Diversifying time series via unique identifiers
public class TimeSeriesIdentifierTransformer {
private final int metricId;
public TimeSeriesIdentifierTransformer(int metricId) {
this.metricId = metricId;
}
public Metric transform(Metric input) {
Metric transformed = input.copy();
// Add a new unique identifier to the labels
transformed.addLabel("metric_id", String.valueOf(metricId));
return transformed; // Generates N-fold cardinality
}
}
Effect:
// M×N Integrated Implementation
for (int timeOffset = 0; timeOffset < M; timeOffset++) {
for (int identifierOffset = 0; identifierOffset < N; identifierOffset++) {
Metric transformed = metric.copy();
// M: Refine the timestamp (increase data density)
transformed.setTimestamp(baseTime.plusSeconds(timeOffset * TARGET_INTERVAL));
// N: Diversify the unique identifier (increase cardinality)
transformed.addLabel("metric_id",
String.valueOf(identifierOffset));
emit(transformed); // Total data volume is M×N
}
}
Final Effect:
Total Data Volume = Original Volume × M × N
Total Cardinality = Original Cardinality × N (M has no effect)
Let’s look at exactly what happens to the database when we turn these dials.
What it stresses:
What we monitor:
- Write throughput (points/sec)
- Disk usage growth rate
- Memory usage for write buffers
- Query response time for time-range queries
What it stresses:
{server="web01_virtual_*"} become much slower.GROUP BY operations must scan N times more series.What we monitor:
- Memory usage for the series index
- Label query performance (milliseconds)
- Cardinality limit warnings
- Query planning time for complex aggregations
With the M×N strategy, we can run targeted scenarios to find exact weaknesses:
Scenario 1: M=60, N=1 → High data density, existing cardinality
Scenario 2: M=1, N=100 → Existing density, high cardinality
Scenario 3: M=10, N=10 → Balanced load test
This lets us find out exactly which part of the TSDB breaks first.
It was important to look past the basic request of “make more data.” The true goal was measuring the performance limits of the TSDB.
Building a system that behaves like the real world is much more valuable than building a theoretically “perfect” but unnatural simulation.
I learned to constantly weigh performance against complexity, and perfection against practicality.
Discussing these ideas with colleagues helped me fix blind spots I never would have seen on my own.
Starting with a simple multiplication idea and slowly refining it worked much better than trying to design a massive, complex system on day one.
This experience taught me how vital it is to understand how data flows over time.
It’s easy to fall into the trap of over-engineering a solution. This process reminded me that the best development involves finding the core problem, matching the real-world environment, and validating ideas with your team.
Part 2 Preview
In Part 2, I’ll explain how we tried to implement this M×N strategy in a real Proof of Concept (PoC) environment.
Spoiler Alert: When I showed this to my team, their feedback led us to scrap the Flink idea entirely. We pivoted to a completely different, much simpler approach. Discover how teamwork turned a complex architecture into a pragmatic solution in Part 2.
All technical content in this article is based on actual production experience. Specific system names and configuration values have been generalized for security.
]]>Recently, while working on a software development project, I had the opportunity to think deeply about Exception Handling.
While integrating a third-party library, I encountered conflicts between the library’s guidelines and our application’s exception handling approach. This led me to reconsider exception handling practices, and I decided to revisit the concepts from Effective Java 3rd Edition - a true bible for Java developers.
As you can see from the table of contents, Exception Handling deserves an entire chapter rather than just a single item, highlighting its critical importance.
Let me explore Effective Java’s exception handling principles and reflect on how to solve the real-world problems I’ve encountered.
The opening statement sets the tone perfectly:
“Used properly, exceptions can improve a program’s readability, reliability, and maintainability. Used improperly, they can have the opposite effect.”
I completely agree with this statement, and I believe developers should always consider whether improper usage might occur.
Exceptions should be used only for truly exceptional situations that disrupt the normal flow of a program.
Using exceptions for situations that can be handled with simple conditional statements leads to performance degradation, reduced readability, and debugging difficulties.
Why shouldn’t exceptions be overused?
Proper Usage Examples
// Wrong: Using exceptions for existence checks in loops
for (String s : list) {
try {
process(s);
} catch (NoSuchElementException e) {
// Simply because the list is empty
// Conditional handling is much more efficient
}
}
// Correct:
for (String s : list) {
if (s != null) {
process(s);
}
}
// File reading where file doesn't exist -> exception appropriate
try {
readFile("nonexistent.txt");
} catch (IOException e) {
System.out.println("Error reading file: " + e.getMessage());
}
Java provides three types of throwables: checked exceptions, runtime exceptions, and errors. Here’s guidance on when to use each:
Recoverable conditions → Checked exceptions Programming errors → Runtime exceptions
RuntimeExceptionChecked exceptions force callers to handle exceptions. However, if it’s not truly a recoverable situation, using checked exceptions hurts API usability and makes code messy.
Why is this problematic?
try {
obj.action();
} catch (SomeCheckedException e) {
// Actually no recovery method available
throw new RuntimeException(e);
}
→ Callers inevitably end up with unnecessary “catch and rethrow” code.
Correct Design Principles
Examples
// Caller cannot actually recover
public void connect() throws IOException {
// IOException thrown on connection failure
}
try {
service.connect();
} catch (IOException e) {
// Cannot recover but must catch and rethrow or just log
}
// Not recoverable → change to runtime exception
public void connect() {
if (/* failure */) {
throw new IllegalStateException("Cannot connect to server");
}
}
// No recovery needed → provide state check method
if (service.canConnect()) {
service.connect();
}
Java provides well-defined standard exception classes. Rather than defining new exception classes, using appropriate standard exceptions is advantageous for consistency, readability, and maintainability.
Standard exceptions should be the first choice, and creating new exceptions should be a last resort.
Why use standard exceptions?
NullPointerException, IllegalArgumentException are self-explanatoryCommonly Used Standard Exceptions
| Exception | Usage |
|---|---|
| IllegalArgumentException | When arguments are invalid |
| IllegalStateException | When object state is inappropriate for method call |
| NullPointerException | When null arguments are not allowed |
| IndexOutOfBoundsException | When index is out of range |
| ConcurrentModificationException | When concurrent modification is prohibited |
| UnsupportedOperationException | When called method is not supported |
Examples
public void setAge(int age) {
if (age < 0) {
throw new IllegalArgumentException("Age cannot be negative: " + age);
}
this.age = age;
}
// Actually IllegalArgumentException is sufficient
public class InvalidAgeException extends RuntimeException {
public InvalidAgeException(String message) {
super(message);
}
}
Exceptions thrown by a method should match the abstraction level of that method.
Lower-level implementation details should not leak through exceptions - they should be translated to exceptions appropriate for the higher abstraction level.
Why is this important?
Wrong Example (Exposing Implementation Exceptions)
// Internal library code
public List<String> readNames() throws SQLException {
// DB access logic
}
SQLException → API changes needed when DB is replacedCorrect Example (Matching Abstraction Level)
// Abstracted API
public List<String> readNames() throws DataAccessException {
try {
// DB access
} catch (SQLException e) {
throw new DataAccessException("Database read failed", e);
}
}
Exception Translation
new MyException("msg", cause))try {
// DB access
} catch (SQLException e) {
throw new DataAccessException("Data access failed", e); // include cause
}
After studying the theory, let me share how I resolved the library integration problem mentioned in the overview.
The third-party library we were integrating validates requests before they reach application controllers. The library guidelines were:
checkPermission() method throws a single AuthorizationException on failureErrorCode indicating the failure reason (TOKEN_EXPIRED, INSUFFICIENT_PERMISSIONS)GlobalExceptionHandler should catch this AuthorizationException, check the internal ErrorCode, and use if/else branching to return different HTTP status codes (401, 403, etc.)[Library-recommended approach]
@RestControllerAdvice
public class GlobalExceptionHandler {
@ExceptionHandler(AuthorizationException.class)
public ResponseEntity<ErrorResponse> handleAuthorizationException(AuthorizationException e) {
// 😫 Logic to analyze the cause inside ExceptionHandler
if (e.getErrorCode() == ErrorCode.TOKEN_EXPIRED) {
return ResponseEntity.status(HttpStatus.UNAUTHORIZED)
.body(new ErrorResponse("Token has expired."));
} else if (e.getErrorCode() == ErrorCode.INSUFFICIENT_PERMISSIONS) {
return ResponseEntity.status(HttpStatus.FORBIDDEN)
.body(new ErrorResponse("Access denied."));
}
// ... various other error code branches
return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR)
.body(new ErrorResponse("Unknown authentication error."));
}
}
However, this conflicted with our project’s exception handling principles:
ExceptionHandler should have simple responsibility - only converting exception types to appropriate HTTP responsesProductNotFoundException, InvalidOrderException)The Effective Java principles I just summarized provided clear direction:
AuthorizationException with library implementation details is inappropriate. TOKEN_EXPIRED should be translated to UnauthorizedException, and INSUFFICIENT_PERMISSIONS to ForbiddenException at a higher abstraction level.Interceptor would act as an ‘adapter’, catching the library’s AuthorizationException and converting it to project-compliant exceptions.1. Define Project-Appropriate Custom Exceptions
First, I defined exceptions appropriate for our project’s abstraction level:
// 401 Unauthorized
public class UnauthorizedException extends RuntimeException {
public UnauthorizedException(String message) { super(message); }
}
// 403 Forbidden
public class ForbiddenException extends RuntimeException {
public ForbiddenException(String message) { super(message); }
}
2. Apply Exception Translation in Interceptor
Then, I modified the AuthInterceptor to catch the library’s exceptions and translate them to appropriate custom exceptions:
@Component
public class AuthInterceptor implements HandlerInterceptor {
private final AuthenticationService authService; // External library service
// ... constructor omitted ...
@Override
public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) {
try {
String token = request.getHeader("Authorization");
authService.checkPermission(token); // This method throws AuthorizationException
} catch (AuthorizationException e) {
// ✨ Exception Translation happens here
if (e.getErrorCode() == ErrorCode.TOKEN_EXPIRED) {
throw new UnauthorizedException("Authentication token is invalid.");
} else if (e.getErrorCode() == ErrorCode.INSUFFICIENT_PERMISSIONS) {
throw new ForbiddenException("You don't have permission to access this resource.");
}
}
return true;
}
}
3. Simplified ExceptionHandler
As a result, the GlobalExceptionHandler returned to its clean original role of ‘simple conversion based on exception type’:
@RestControllerAdvice
public class GlobalExceptionHandler {
// 👍 Handler with clear roles and responsibilities
@ExceptionHandler(UnauthorizedException.class)
@ResponseStatus(HttpStatus.UNAUTHORIZED)
public ErrorResponse handleUnauthorized(UnauthorizedException e) {
log.info(e.getMessage());
return new ErrorResponse(e.getMessage());
}
@ExceptionHandler(ForbiddenException.class)
@ResponseStatus(HttpStatus.FORBIDDEN)
public ErrorResponse handleForbidden(ForbiddenException e) {
log.warn(e.getMessage());
return new ErrorResponse(e.getMessage());
}
// ... other handlers ...
}
While library and framework guidelines are important, when they conflict with our application’s overall design principles and consistency, it may be better to integrate them non-intrusively by adding an ‘adapter’ layer as shown above.
Ultimately, good exception handling goes beyond simply catching errors - it’s an important design activity that enhances code readability, maintainability, and overall system stability.
This article is based on real work experiences, with specific system names and configuration values generalized for security purposes.