Skip to content
View rohankumardubey's full-sized avatar
:octocat:
Focusing
:octocat:
Focusing
102 followers · 1.7k following

Achievements

Achievement: Pair Extraordinairex3Achievement: Pull Sharkx3Achievement: Galaxy BrainAchievement: QuickdrawAchievement: YOLO

Achievements

Achievement: Pair Extraordinairex3Achievement: Pull Sharkx3Achievement: Galaxy BrainAchievement: QuickdrawAchievement: YOLO

Organizations

@StreamNest

Block or report rohankumardubey

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
rohankumardubey/README.md

Rohan Dubey — Staff Data Engineer & Staff Platform Engineer. Data platforms. Distributed systems. Built with intention.

Portfolio ↗   /    LinkedIn ↗   /    Get in touch ↗


I’m Rohan, a Staff Data Engineer & Staff Platform Engineer working at the intersection of distributed systems, real-time data, and performance engineering. I build platforms with an emphasis on reliability, clear operational behavior, and the details that make them fast.

This is where I explore those ideas in code: stream processing semantics, columnar query execution, and the tradeoffs behind data-intensive systems.

Good infrastructure earns trust quietly: observable, recoverable, and predictably fast.


Selected work

Four projects exploring how data moves, how engines execute, and where performance comes from.

Dremel — Rust and C++ columnar SQL engines. Explore the repository. SpiderOxide — an asynchronous Python crawler with Rust acceleration. Explore the repository.

DataWizz — a local-first lakehouse and analytics workspace. Explore the repository. FlowCore — a Rust stream processing engine with event time and checkpoints. Explore the repository.

All repositories ↗

Engineering focus

Discipline The details that matter
Stream processing Event time, watermarks, backpressure, checkpoints, and recovery.
Query engines Columnar formats, vectorized execution, memory layout, and honest benchmarks.
Distributed data Partitioning, consistency, fault tolerance, and lakehouse architecture.
Platform engineering Kubernetes, infrastructure as code, observability, and developer experience.

Tools of the trade

Languages   Rust · Go · Java · Python · Scala · C++

Data systems   Kafka · Flink · Spark · Arrow · DataFusion · Iceberg · ClickHouse · Trino · DuckDB

Infrastructure   Kubernetes · Docker · Terraform · AWS · Airflow · Prometheus · Grafana

Across the wider stack
Layer Technologies
Messaging Apache Pulsar, Kafka Streams, RabbitMQ, NATS
Compute Spark Structured Streaming, Ray, Apache Hadoop
Analytics Presto, Dremio, Apache Druid, Apache Pinot
Storage & formats Delta Lake, Apache Hudi, Parquet, MinIO
Databases & search PostgreSQL, Cassandra, ScyllaDB, CockroachDB, TiDB, MongoDB, Redis, Elasticsearch
Cloud & orchestration Snowflake, BigQuery, Databricks, Argo

Built in the open

I learn by reading implementations, testing ideas against real workloads, and contributing fixes and improvements. My interests keep bringing me back to data engines, streaming systems, storage, and Rust infrastructure.

Explore my open source contributions ↗

GitHub stats

Rohan Dubey's GitHub statistics: stars, commits, pull requests, issues, and contributions Rohan Dubey's most used languages by repository code size

Contribution streak

Rohan Dubey's total contributions, current streak, and longest contribution streak

Contribution signal

Rohan Dubey's daily GitHub contributions over the last 31 days

Refreshed daily · View live GitHub activity ↗


Always up for a thoughtful conversation about systems, data, and the craft of engineering.

Let’s build something that lasts ↗

Pinned Loading

  1. flink flink Public

    Forked from apache/flink

    Apache Flink

    Java

  2. Dremel Dremel Public

    A Rust columnar SQL engine inspired by the Dremel paper, with Arrow and Parquet support and a C++ reference for benchmarking.

    Rust

  3. spark spark Public

    Forked from apache/spark

    Apache Spark - A unified analytics engine for large-scale data processing

    Scala 2

  4. ClickHouse ClickHouse Public

    Forked from ClickHouse/ClickHouse

    ClickHouse® is a real-time analytics database management system

    C++

  5. tidb tidb Public

    Forked from pingcap/tidb

    TiDB - the open-source, cloud-native, distributed SQL database designed for modern applications.

    Go

  6. SpiderOxide SpiderOxide Public

    Rust-accelerated Python crawler for structured web data and AI pipelines, with Scrapy-style APIs and persistent crawls.

    Python 1