Conversation
Tom-Willemsen
left a comment
Tom-Willemsen
left a comment
There was a problem hiding this comment.
I'll give this a more thorough functional test tomorrow...
src/howl.rs
Outdated
|
|
||
| let start_time = SystemTime::now() | ||
| .duration_since(SystemTime::UNIX_EPOCH) | ||
| .unwrap() |
There was a problem hiding this comment.
Just generally: please try to avoid bare unwrap() - it gives almost no context.
Prefer .expect("some reason why this can never fail") for truly infallible operations, or Result for anything that can legitimately fail.
| fbb.finished_data() | ||
| } | ||
|
|
||
| #[allow(clippy::too_many_arguments)] |
There was a problem hiding this comment.
I know this is pretty much exactly how I'd written it in the python too, with a zillion arguments, but I think the more extensible approach is to pull all of the "howl config" arguments into a HowlConfig struct, instantiated once in main, and passed through as &HowlConfig to anywhere that needs it.
This avoids multiple methods all taking a similar huge list of arguments.
| mut current_job_id: String, | ||
| ) -> String { |
There was a problem hiding this comment.
If you take a mutable reference you could avoid needing to return the new String here.
| println!("Each ev44 is {ev44_size} bytes"); | ||
|
|
||
| let producer: ThreadedProducer<DefaultProducerContext> = ClientConfig::new() | ||
| .set("bootstrap.servers", broker) |
There was a problem hiding this comment.
How fast were you able to howl with just this?
There was a problem hiding this comment.
I think it was doing 200MB/s on a single partition but will double check.
There was a problem hiding this comment.
Ok - interesting - I found I needed a bit of Kafka config to actually go that fast in aggregator...
Co-authored-by: Tom Willemsen <tom.willemsen@stfc.ac.uk>
2 participants
Convert saluki to rust.
Changes to note:
listenandconsumehave been merged, as they were quite similar anyway. README and--helpshould clarify how to useconsume(which is aliased tolistenfor muscle memory's sake!)playhas not been carried over as I ran out of time to be able to call this personal development. rewriteplayin rust #50 exists to add it when we need to use it again.howlis a bit faster than python, though I haven't tried the "cheat" way of only constructing one set of det values but sending them x times a frame.also note that the CI is currently failing at the time of writing but only because one of the apt mirrors seems to be 404ing?working nowI may also need a hand benchmarking
howlplz @Tom-Willemsen