Make hashing cheaper on hot paths - #85
Conversation
Motivation: Hasher.init and Hasher.finalize show up heavily in QUIC benchmarks (~4-5% of cycles for 1KB download for 100 concurrent streams). There are three main sources for this in swift-network: 1. The transport parameters map, 2. The knownFlows map in QUICConnection, and 3. the MultiplexedFlowIdentifier lookup. The transport parameters stores at most 19 elements and is keyed by an enum: this alone is just shy of ~1% of the cycles. The known flows lookup is also around ~1%, while the multiplexed flow ID accounts for the rest. Modifications: - Back TransportParameters by an array of optionals instead of a dictionary: the transport parameters types are each assigned a fixed index in the array. This eliminates hashing there altogether at the cost of ever so slightly more memory. - Key the known flows map in QUICConnection by the raw value for the stream ID instead of the nominal type. Swift special cases the hashing of various types to elide the creation of Hasher. - Implement the '_rawHashValue' customisation point in MultiplexedFlowIdentifier: this also avoids creating a Hasher. - The hash computation of MultiplexedFlowIdentifier also uses two bits as a discriminator for the underlying case. Result: The first two optimisation increase throughput in 1KB download QUIC benchmarks by ~1.9%. Adding the multiplexed flow identifier change in raises that to ~4.9%.
| public func _rawHashValue(seed: Int) -> Int { | ||
| self.rawHashKey._rawHashValue(seed: seed) | ||
| } |
There was a problem hiding this comment.
I would love to get @lorentey's opinion on the use of this particular API since it's underscored...
There was a problem hiding this comment.
This is a great use of _rawHashValue! Forwarding to UInt64's implementation still ensures strong hashing, and marking the case in the two lower bits helps reduce collisions in the original implementation. 👍
| var pendingReassemblyDequeue = QUICStreamList.pendingReassemblyDequeueList() | ||
|
|
||
| private(set) var knownFlows = [QUICStreamID: MultiplexedFlowIdentifier]() | ||
| // The logical key choice is 'QUICStreamID', however, Swift special cases the hashing of |
There was a problem hiding this comment.
@glbrntt what if we just made QUICStreamID be RawRepresentable and just be a raw UInt64? That's all it stores anyway.
There was a problem hiding this comment.
I looked into that: sadly RawRepresentable hits the slow path! The default implementation of _rawHashValue(seed:) is this:
public func _rawHashValue(seed: Int) -> Int {
var hasher = Hasher(_seed: seed)
hasher.combine(self)
return hasher._finalize()
}
So for RawRepresentable to get the fast path of not creating the Hasher it would have to forgo allowing combine(_:) to be customized.
There was a problem hiding this comment.
We may want to consider typealias QUICStreamID = UInt64 and then deal with the methods as extensions to UInt64 to avoid more problems like this?
| // The logical key choice is 'QUICStreamID', however, Swift special cases the hashing of | ||
| // various primitives (by avoiding the construction of a Hasher altogether). The result is | ||
| // that hashing the raw value is significantly cheaper which adds up on hot paths. | ||
| private(set) var knownFlows = [UInt64: MultiplexedFlowIdentifier]() |
There was a problem hiding this comment.
Nice, thank you for uncovering this @glbrntt !
| @_spi(Essentials) @_spi(ProtocolProvider) @testable import SwiftNetwork | ||
|
|
||
| @available(Network 0.1.0, *) | ||
| final class SwiftNetworkTransportParametersTests: XCTestCase { |
There was a problem hiding this comment.
Shouldn't this test be part of the QUICTests instead?
5 participants
Motivation:
Hasher.init and Hasher.finalize show up heavily in QUIC benchmarks
(~4-5% of cycles for 1KB download for 100 concurrent streams). There are
three main sources for this in swift-network:
The transport parameters stores at most 19 elements and is keyed by an
enum: this alone is just shy of ~1% of the cycles. The known flows
lookup is also around ~1%, while the multiplexed flow ID accounts for
the rest.
Modifications:
dictionary: the transport parameters types are each assigned a fixed
index in the array. This eliminates hashing there altogether at the
cost of ever so slightly more memory.
stream ID instead of the nominal type. Swift special cases the hashing
of various types to elide the creation of Hasher.
MultiplexedFlowIdentifier: this also avoids creating a Hasher.
as a discriminator for the underlying case.
Result:
The first two optimisation increase throughput in 1KB download QUIC
benchmarks by ~1.9%. Adding the multiplexed flow identifier change in
raises that to ~4.9%.