Kotlin Sequences vs Collection Operators: Impact Of Data Distribution

I recently read Chris Banes’ post “Should you use Kotlin Sequences for Performance?” and found his findings intriguing, especially when compared with Max Sidorov’s results in “Measuring sequences” and “Kotlin under the hood: how to get rid of recursion”. The differences in environments, methodologies, pipelines, and data produced varying benchmark results, leading to potential misunderstandings. In this post, I’ll show how data distribution can significantly impact the performance of processing pipelines under different parameters.

Before moving on, there’s a crucial reminder from JMH (Java Microbenchmark Harness):

REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial experiments, perform baseline and negative tests that provide experimental control, make sure the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts. Do not assume the numbers tell you what you want them to tell.

What Are We Really Comparing?

When comparing Kotlin Sequences and Collection Operators, it’s important to understand that we’re not merely comparing abstract concepts like pull/push models or lazy/eager evaluation. Instead, we’re examining how these approaches perform in concrete scenarios and pipelines.

Eager evaluation on bare loops is a common pattern, but lazy calculations are not. If you’re not familiar with the differences between these approaches, I recommend checking the official Kotlin documentation for diagrams illustrating how Sequence operators differ in execution, as well as Max Sidorov’s “Sequences: Theory” for additional explanation.

The Pipeline Structure

I liked Chris’s approach of using a simple repository example for benchmarking, so I’ve reused that pattern:

class SimplePipelineRepository(private val db: Db) : DataRepository {
    override fun getItemsListCollectionOperators(): List<UiModel> {
        return db.getItems()
            .filter { it.isEnabled }
            .map { UiModel(it.id) }
    }

    override fun getItemsListSequence(): List<UiModel> {
        return db.getItems()
            .asSequence()
            .filter { it.isEnabled }
            .map { UiModel(it.id) }
            .toList()
    }
}

These pipelines have identical named operations and lambda code. But if the “work” is the same, what exactly are we comparing? The benchmark results for these pipelines will include:

• Method calling overhead (hasNext/next on filter and map) during one cycle versus two cycles with “flat” calculations
• Memory management differences between the approaches
• How the JIT compiler optimizes each pipeline
• Hardware performance characteristics

For those interested in benchmarking JVM apps, I highly recommend the JMH Samples as the best resource for learn how to write benchmarks . Some samples demonstrate how to avoid or minimize the impact of JIT optimizations.

But for this post I don’t use them. :)

Setting Up

Benchmark Parameters

To explore how Sequences and Collection Operators behave across different scenarios, I’ve built a matrix of benchmarks varying three key parameters:

1. Collection size (batch size): The number of items in the input list
2. Filtering ratio (percent): The percentage of items that will pass the filter logic - 0%, 10%, 25%, 50%, 75%, 90%, and 100% (from 0.0 to 1.0)
3. Data distribution: How filtered items are arranged in the input list
   • Ordered: Most passing items are at the beginning of the list
   • Distributed: Passing items are distributed with fixed steps throughout the list
   • Shuffled: Items are randomly arranged using Kotlin’s default shuffle algorithm

Environment

For all tests, I used the following environment:

• Hardware: MacBook Pro M1 Pro (8-core) with 32GB RAM
• OS: macOS Sequoia 15.3.2
• JDK: Temurin-21.0.6+7
• JMH: version 1.37
• Kotlin: 2.0.20

Benchmarking Medium and Large Collections

In this section, I’ll present results for collection sizes of 100, 1,000, 10,000, and 100,000 elements. For the medium and large batch sizes, I used this configuration:

• Warm-ups: 10
• Measurements: 25
• Forks: 2
• Mode: Throughput
• Duration: 1 second
• GC: G1
• Fixed Heap size: 2GB
• Enabled +AlwaysPreTouch option to ensure memory is fully allocated at startup to avoid performance spikes

If you want to measure time, be careful—especially for very fast computations. The ancient manuscript “Nanotrusting the Nanotime” by Aleksey Shipilёv explains it in detail.

In the tables below, the “Diff %” column shows the percentage difference between Collection Operators (ColOps) and Sequences implementations. Positive values (+%) indicate Sequences are faster, while negative values (-%) indicate Collection Operators are faster.

Ordered data

The table is based on data from [CSV ].

Key Observations for Ordered Data:

• With ordered data, Sequences show a dramatic advantage (+82.6%) for small collections with low filter percentages (10%)
• As the filter percentage increases, the advantage diminishes and sometimes reverses
• For medium-to-large collections (10,000+), Sequences perform better at very low (0-10%) and very high (90-100%) filter percentages

Shuffled data

Yes, it’s strange to use shuffled data, but it’s the easiest way to demonstrate behavior in an “unstable” state.

The table is based on data from [CSV ].

Key Observations for Shuffled Data:

• High filter percentages (90-100%) show more consistent results regardless of distribution

Distributed data

The table is based on data from [CSV ].

Key Observations for Distributed Data:

• Distributed data produces results similar to shuffled data for many cases
• For small collections with low filter percentages (10%), Sequences perform even worse
• As collection size increases, the distribution pattern impacts performance more significantly

Comparative Analysis of Distribution Impact

To better understand how data distribution affects each approach, I’ve compared the same implementation across different distribution patterns.

Impact on Collection Operators

Key Observations: Collection Operators show relatively low sensitivity to data distribution for small and medium collections. For the largest collections (100,000 items), however, shuffled data can cause dramatic performance drops (up to 52% slower compared to ordered data).

Impact on Sequences

Key Observations: Sequences show significantly higher sensitivity to data distribution:

• For 10% filter with batch size 100, changing from ordered to distributed data causes a dramatic 72.41% performance drop
• For larger collections (100,000 items), distribution can impact performance by over 58%
• Sequences maintain consistent performance regardless of distribution when the filter percentage is very high (90-100%)

Summary of Medium and Large Collection Analysis: Collection Operators are generally less affected by data distribution than Sequences. The impact of distribution becomes more pronounced for both approaches as collection size increases, with Sequences showing greater sensitivity to distribution patterns when dealing with low filter percentages.

The Case of Small Collections

For smaller collection sizes, I ran the benchmark with a different configuration:

• Warm-ups: 10
• Measurements: 10
• Forks: 2
• Mode: Throughput
• Duration: 500ms
• GC: G1
• Fixed Heap size: 2GB
• +AlwaysPreTouch: Ensure memory is fully allocated at startup

Please note that results for batch size 5 should be interpreted with caution, as the filtering logic may not be ideal for such small collections.

The table is based on data from [CSV ].

Key Observations for Small Collections:

• For very small collections (5-10 items), Sequences consistently outperform Collection Operators across all filter percentages
• The advantage is most pronounced at low filter percentages (58-67% faster for 0-25% filters)
• As collection size increases to 50+, the performance pattern starts to resemble that of medium-sized collections
• For batch size 25, we observe higher variability in results, suggesting this might be a transitional size where JIT optimization patterns change

Summary of Small Collection Analysis: Very small collections (under 25 items) show a distinctly different pattern from medium and large collections. Sequences consistently outperform Collection Operators for these tiny collections, regardless of filter percentage or distribution pattern. This advantage diminishes as collection size increases, eventually transitioning to the more complex pattern observed with medium-sized collections.

Conclusion

Comparing Sequences and Collection Operators is not straightforward. Data distribution can impact performance, sometimes reversing the performance advantage between the two approaches. JIT and Hardware optimizations do a lot of work. Really. All this results are heavy depend on Branch predictor implementation and how particular JIT and Hardware work with them in case of Sequences and Collection operators.

Based on all results we can conclude:

1. Distribution matters: The arrangement of passing items in your collection can affect performance by up to 72% in some cases.
2. Collection size is crucial: Very small collections behave differently from medium and large ones.
3. Filter percentage affects sensitivity: Low and high filter percentages show different sensitivity to distribution patterns.
4. No universal winner: Neither approach consistently outperforms the other across all scenarios.

Rather than making a blanket recommendation, it’s more valuable to understand the factors that influence performance in your specific use case. Consider:

• What is your typical collection size?
• What percentage of items typically pass your filter?
• Is your data likely to have a specific distribution pattern?
• How critical is performance for this particular operation?
• Do you need a more stable results?

Importand update

For Android devs: this results isn’t fit for an Android apps. The environemt is different, AOT and hardware different. Don’t try to apply it to your apps directly, but pay attention on your benchmarks.

P.S.: In future posts, I plan to explore:

• Applying JMH samples recommendations to benchmarks and showing how results change, checking how it would impact Android
• Finding a “baseline” for comparing Sequences and Collection Operators
• Comparing memory usage between the two approaches