Skip to content

actor: init - #463

Merged
npry merged 41 commits into
mainfrom
npry/actor
Nov 5, 2026
Merged

npry merged 41 commits into
mainfrom
npry/actor

Conversation

@npry

@npry npry commented Oct 28, 2026 •

Copy link
Copy Markdown
Member

This is an actor system aiming to replace our usage of kameo in ts_runtime, due to:

  • Lack of expressiveness and in-band soundness issues on kameo's part (no process monitors, pid reuse, baked supervisors, poor support for oneshot tasks)
  • Maintainer commitment to tokio specifically, no realistic possibility of #![no_std]
  • Ease of adaptation from having our own system in-tree
    • Coupled with the last point, it may be relevant for us to have the ability to pretty tightly control allocation, which isn't aligned with kameo's goals as I understand them
  • Unpartitionable global state (no sub-scoping or notion of starting "an actor system", just one big actor universe per OS process)

ts_actor sticks pretty close to Erlang semantics, preferring to surface the fact that all message passing must involve dynamic types under the hood; messages must be downcast on receipt. Compared to other Rust actor systems, this permits trait-less lightweight processes with similar semantics to an async executor's tasks, but where each task (following Erlang, called a "process") is augmented with a ProcContext which provides access to runtime-global functions and state (comparable to Erlang bifs and standard library functions), such as sending and receiving messages, spawning new processes, looking up other processes, etc. Running a process can look as simple as e.g.:

runtime.spawn(async |ctx: ts_actor::ProcContext| {
    ctx.send(&some_other_pid, ts_actor::message![4u8]).await?;

    let resp = ctx.recv().await?;
    let msg = resp.downcast::<String>()?;
    println!("msg");

    Ok(())
});

The runtime is not tightly coupled to a particular async executor, and only requires task spawning and a sleep function at the moment, which are accessed through a dynamically dispatched Executor trait. Spinning up a runtime just looks like:

// Tokio implements ts_actor::runtime::Executor
let rt: Arc<dyn Rt> = ts_actor::runtime::new(Tokio::current());

There are executors provided for tokio and async-executor (smol ecosystem).

The system does take substantial inspiration from kameo re: what is useful in a Rust actor system, so there's a typed message bus (with retained message support) and utilities to forward a stream to an actor or onto a bus. Plenty is taken from erlang as well, so it provides a process registry, a supervisor module, and gen_server.

Genservers are in a somewhat interesting place as they are kind of "gradually typed" — the base trait just requires you to downcast incoming messages yourself, but you can provide TypedCall<T>, TypedCast<T>, and TypedInfo<T> for the genserver and then generate handle_{call,cast,info} using a macro, which handles the down/upcasting for you automatically. I'm gesturing at gradual typing because the guarantees are relatively weak and you have to opt in.

As an aside, I found myself very naturally falling into/rediscovering the Erlang/Elixir pattern of module-scoped genservers which only expose api functions on &ProcContext to talk to them; the fact that the module runs a genserver at all is totally opaque. Something like this:

// calculator.rs

// --- api ---
pub async fn start(ctx: ProcContext) -> Result<() ,Error> {
    ctx.register(&NAME)?;
    gen_server::start(Calculator, ())(ctx).await
}

pub async fn add(ctx: &ProcContext, a: u32, b: u32) -> Result<u32, ts_actor::Error> {
    gen_server::call_typed::<_, Calculator>(ctx, &NAME, Add(a, b)).await
}

// --- implementation ---
// note that nothing outside this file needs to know about the types needed to implement the api
// or see the genserver implmentor `Calculator`, they can all be private because the api functions are all that
// needs to see them. this keeps the scope of the distributed systems problem kind of limited; we can see
// all the possible entrypoints for anything to talk to this implementation in one place

const NAME: SmolStr = SmolStr::new_static("calc");

struct Calculator;

struct Add(u32, u32);

impl GenServer for Calculator {
    type State = ();
    type Args = ();

    // init elided

    typed_call!(Add);
}

impl_call!(Calculator, Add, u32, ctx, state, msg, handle, {
    handle.commit(msg.0 + msg.1).into()
});

Review notes

The first few commits are components of the system that could be broken out independently; they're used later. Worth paying attention to especially oneshot (has unsafe) and mailbox.

The "core runtime" commit has the meat of the implementation under the runtime module. Technically gen_server and registry are needed as part of that, since the global process registry itself runs as a process, which is a genserver, but this is mostly conceptually separable from the runtime module.

After that is library functionality which requires the runtime to be implemented.

I'm still working on porting ts_runtime — that's in a local branch for now.

@npry
npry force-pushed the npry/actor branch 7 times, most recently from 4bc871e to 721d3b8 Compare October 28, 2026 21:29
@npry
npry marked this pull request as ready for review October 28, 2026 21:51

@nrc nrc left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've just been browsing the code to figure things out, this initial review is basically just nits I noticed while doing a random walk.

#[cfg(feature = "tokio")]
pub use _tokio::*;

#[cfg(feature = "smol")]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is worthwhile holding Tokio via an abstraction, but I don't think it is worth it to support smol/async_executor. If the point is to prove that it can work with other runtimes, then I think supporting something more radically different would be better (e.g., glommio)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, why not re: async-executor? Just too similar? Lost the ecosystem war? glommio might be interesting but seems kind of dead (no commits for ~18 months).

Partially the idea was also to get the task abstraction right, since they're meaningfully different shapes -- async-executor just forwards the panic when you .await the task, while tokio catches it, so there's something meaningful there that I do want to ensure works across different executors

Comment thread crates/utils/ts_actor/src/lib.rs Resolved
Comment thread crates/utils/ts_actor/src/lib.rs Resolved
Comment thread crates/utils/ts_actor/src/lib.rs Resolved
Comment thread crates/utils/ts_actor/src/message.rs Resolved
@@ -0,0 +1,401 @@
//! Adaptation of Erlang's `gen_server`: processes that send and receive messages like RPCs, i.e.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realise it's an Erlang-ism, but I find the name pretty confusing. Could we use server or process or actor? (I'm not clear on the exact difference between those concepts).

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm leaning towards renaming this Server since I've heard complaints from both you and Dylan about this -- will hold off on this PR and look at it after

@@ -0,0 +1,205 @@
//! Experimental concretization of owned, `'static` lifetime storage, allowing either `Box` or

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I'd put this in its own crate

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

generally 👍 but am going to do this in a follow-up

msg: Message,
}

struct Cast(Message);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, it's an Erlang-ism, but I find 'cast' to be really confusing - especially since downcasting is common

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto, considering what/whether to rename but will hold off for this PR

@npry
npry force-pushed the npry/actor branch 2 times, most recently from db2a2e7 to cdd2ab8 Compare October 29, 2026 00:21

@nrc nrc left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A bunch more comments, sorry for the piece-meal review but I have to stop for the day and I figure it's better to send what I've got. Sorry if anything is in correct

Comment thread crates/utils/ts_actor/src/message.rs Resolved
/// assert!(val);
/// ```
#[macro_export]
macro_rules! msg_match {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't really explain why, but I'm not a fan of this msg_match macro. I think because for me it is in the uncanny valley between low-level and high-level abstraction. I assume that the main use case is going to be in message handlers, so I wonder if with a bit of macro magic we could come up with some kind of dispatch macro that takes us from dynamically to statically typed in a higher-level (or maybe just more OO) way?

@npry npry Oct 29, 2026 •

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's chat about this at our sync today -- I guess I'm not sure what shape you'd be looking for; the macro does take you from dynamically to statically typed in a concise way

Global(SmolStr),
/// A name registered in a given registry. It will be resolved using the generic
/// registry protocol.
Via {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this necessary? Seems like quite niche functionality

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Necessary no, but there is utility. You'd want it if you're writing a module that doesn't want to coordinate naming with the global registry, either because it can't (genuinely might collide / there's code you don't control which might take arbitrary names) or it's just more convenient to do scoped names. So for us e.g. we could have the Multiderp actor serve as a registry for the uniderps -- their names would be their region id only and not something like uniderp-{region}. On our scale/with the current scope of the actor system it's overkill I think because we can "see" all the naming decisions and just not cause collisions, so yeah, I think we can remove.

Comment thread crates/utils/ts_actor/src/actors/bus.rs Resolved
Comment thread crates/utils/ts_actor/src/actors/bus.rs Resolved
Comment thread crates/utils/ts_actor/src/runtime/rt.rs Resolved

@nrc nrc left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another batch of review comments. I have reviewed everything except the gen_server and actors modules. Looks good!

Comment thread crates/utils/ts_actor/src/oneshot.rs Resolved
Comment thread crates/utils/ts_actor/src/oneshot.rs Resolved
Comment thread crates/utils/ts_actor/src/oneshot.rs Resolved

/// Configure this process to trap exits from linked processes.
///
/// This causes the runtime to send a

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this different from monitoring?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, they're used in different contexts -- links are bidirectional and 1:1, while monitors are unidirectional and one-to-many. You want a link if you wanted to express a tight process coupling, and you want to trap_exit if you need to do something other than die immediately when the linked process goes down. You still want the link here because you want the other process to die if this one goes down.

Monitors are useful in scenarios when you just need to know about changes to another process's state without expressing that unique dependency coupling. You might use them to clean up state or cancel an in-flight request -- in these scenarios the monitored process actively shouldn't be killed if we die.

Comment thread crates/utils/ts_actor/src/context.rs Resolved

crate::impl_info!(Supervisor, SystemMessage, ctx, state, msg, {
match msg {
SystemMessage::LinkExit {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit confused about how the supervisor fits in with monitors/links and the runtime's monitoring/propagation of killed tasks. I think it would benefit from a bit more documentation.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto re: below comment

};

let start = spec.start.clone();
let pid = ctx.spawn_link(move |ctx| start(ctx))?;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a bit of a vague comment, but it seems the startup state of the supervisor and its child processes is a bit murky/complex. (e.g., there can be entries in specs but not the other maps, it can be running or not, etc.). It probably just needs the invariants at supervisor startup documented, or perhaps a state diagram or something.It would be great if it could be simplified, but it looks like the situation is deliberate.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is a bit complex implementation-wise, but I think it shouldn't feel that way -- the children that are there at startup are basically treated as if they were all dynamically spawned. Each of them has their own lifecycle state basically, which depends upon the supervisor's restart policy and some child configuration -- the supervisor just tries to restart them on a timer and may eventually give up and die itself. This is somewhat underbaked right now, so I'm intending to document as part of a follow-up

@nrc nrc left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! I've got a lot of comments but I think they are all pretty minor. I would like to see if we could make it a bit more Rust-idiomatic/ergonomic, but I think it's fine to experiment in-tree and I don't have any concrete ideas.

///
/// The difference between this and [`GenServer::handle_info`] is that cast messages are
/// deliberately sent via [`GenServer`] machinery, while info messages are a catchall
/// for everything else.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not clear on why this is a difference or what the significance of the difference is?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll take a todo on this, I think this is guide-level documentation rather than inline

@npry
npry force-pushed the npry/actor branch 2 times, most recently from 959ad02 to 7fd1d4d Compare November 4, 2026 18:18
npry added 9 commits November 5, 2026 14:12
Signed-off-by: Nathan Perry <nathan@tailscale.com>
Change-Id: Ibd845aececf85ffc665545717770ec036a6a6964
Signed-off-by: Nathan Perry <nathan@tailscale.com>
Change-Id: I33d026224ba786f671bd504f37437c2d6a6a6964
Signed-off-by: Nathan Perry <nathan@tailscale.com>
Change-Id: Ibaa823b31f5236db856556f0de3344386a6a6964
Signed-off-by: Nathan Perry <nathan@tailscale.com>
Change-Id: I2169f9e9e7d3367a7300eb4e389d930b6a6a6964
Signed-off-by: Nathan Perry <nathan@tailscale.com>
Change-Id: Ia2b0ec2c9b5ba94aae34fe210cca0ed46a6a6964
Signed-off-by: Nathan Perry <nathan@tailscale.com>
Change-Id: Iaa5dccb76b10f489b0e7999d7eff662d6a6a6964
This is the major change that implements the core actor system runtime.
Most of the component changes here are interdependent so this commit
can't be meaningfully split out.

Signed-off-by: Nathan Perry <nathan@tailscale.com>
Change-Id: I512ad5ca745602cbec4e8819413d6a846a6a6964
Signed-off-by: Nathan Perry <nathan@tailscale.com>
Change-Id: I512ad5ca745602cbec4e8819413d6a846a6a6964
Signed-off-by: Nathan Perry <nathan@tailscale.com>
Change-Id: Ibd845aececf85ffc665545717770ec036a6a6964
npry added 27 commits November 5, 2026 14:12
Signed-off-by: Nathan Perry <nathan@tailscale.com>
Change-Id: Ic13bc0072d6f8f9a62b25ca546d614176a6a6964
Signed-off-by: Nathan Perry <nathan@tailscale.com>
Change-Id: Ic13bc0072d6f8f9a62b25ca546d614176a6a6964
Signed-off-by: Nathan Perry <nathan@tailscale.com>
Change-Id: Ic13bc0072d6f8f9a62b25ca546d614176a6a6964
This removes the need to check for a full mailbox and makes sending
messages much simpler. The caller can now always send, and the only
error they're going to run into is a noproc.

This is also consistent with Erlang default behavior.

Signed-off-by: Nathan Perry <nathan@tailscale.com>
Change-Id: I0e7e75ee5d0774b3b63363773e6180816a6a6964
Signed-off-by: Nathan Perry <nathan@tailscale.com>
Change-Id: I8b68a969f6fce36b159e71e50d9a8cd66a6a6964
Signed-off-by: Nathan Perry <nathan@tailscale.com>
Change-Id: I1cdb40b5417e57ca23db9831c7afc1266a6a6964
Signed-off-by: Nathan Perry <nathan@tailscale.com>
Change-Id: Ia92b3e4c0b5ba2ab853cf3ae9d427a816a6a6964
Signed-off-by: Nathan Perry <nathan@tailscale.com>
Change-Id: Iac53fab2167f3ea8c9969cc0bb0053d36a6a6964
The check in Inner::remove_one was looking for whether the ptable was
empty, which was desynced with the one in wait_for_proc, which checks
that the ptable is empty or the only process is the global registry.
Refactor the check out to a helper function and use in both places.

Signed-off-by: Nathan Perry <nathan@tailscale.com>
Change-Id: Iae828b28c27c4928ca2cabdbacfb380c6a6a6964
This enables a process to monitor another more than once. This can
be needed e.g. if user logic has a general liveness monitor while
library functionality needs a scoped monitor while performing a specific
message exchange, running a timeout, etc.

This brings monitor behavior into alignment with the Erlang model (just
missed in the original implementation).

Signed-off-by: Nathan Perry <nathan@tailscale.com>
Change-Id: I644518c3a267869e6c0d91f6de604d876a6a6964
This is an RAII type to acquire a monitor which is removed when the
scope closes. We will use this in e.g. `gen_server::call` to abort the
call if the remote genserver process dies.

Signed-off-by: Nathan Perry <nathan@tailscale.com>
Change-Id: I2a4c0758ec23795a219fc2beeab315166a6a6964
Acquire a monitor before starting a gen_server::call and listen for the
incoming Down message in tandem with the reply. The GenServer dying
causes the call to return noproc.

Signed-off-by: Nathan Perry <nathan@tailscale.com>
Change-Id: I864d25bd021d943b70261150556e6ee16a6a6964
Rename ReplyHandle::reply to send_reply to indicate that you're sending
the reply now. This was more confusing previously, as the commit()
method _produces_ a Reply struct. Still not perfect, but an improvement.

Signed-off-by: Nathan Perry <nathan@tailscale.com>
Change-Id: I9fffdd66237a42b4502023303f1a2def6a6a6964
Change `RuntimeInner::remove_one` to return a bool reporting whether the
named process existed, rather than a `Result`.
`RuntimeInner::remove_proc` reports only whether the passed pid existed
(also via bool), and doesn't stop if one of the links is dead.

Signed-off-by: Nathan Perry <nathan@tailscale.com>
Change-Id: I5abb51ffb5d455d074483b0bd81b0ebe6a6a6964
Add `None` variant to `Action`, make returns all non-`Option`.

Signed-off-by: Nathan Perry <nathan@tailscale.com>
Change-Id: Iea13ad718d617d5ffe96ebe8b71fe4a76a6a6964
Signed-off-by: Nathan Perry <nathan@tailscale.com>
Change-Id: I519ae8d0bac6c681985352900d79df886a6a6964
Signed-off-by: Nathan Perry <nathan@tailscale.com>
Change-Id: I026b0fc02bbaf065c88870c262edcc026a6a6964
Signed-off-by: Nathan Perry <nathan@tailscale.com>
Change-Id: I50330eeecf045b8bcc08d331d611db106a6a6964
Signed-off-by: Nathan Perry <nathan@tailscale.com>
Change-Id: Iaf7b98aa4cb004543bb142bed386aeb56a6a6964
Signed-off-by: Nathan Perry <nathan@tailscale.com>
Change-Id: Ife8e4c4836b4eb29f1f5c382f75b52e26a6a6964
Signed-off-by: Nathan Perry <nathan@tailscale.com>
Change-Id: Ib43d7b76bb038e06f582f66b6814ae456a6a6964
Signed-off-by: Nathan Perry <nathan@tailscale.com>
Change-Id: I87e72dd7de396b7de66387d8631352876a6a6964
Signed-off-by: Nathan Perry <nathan@tailscale.com>
Change-Id: I645f2fdd97f099cfe1962fab81ae76a96a6a6964
Signed-off-by: Nathan Perry <nathan@tailscale.com>
Change-Id: Ie78cc7b6b03fe5972ebb57ae04e66f5a6a6a6964
Add specific error cases which `impl Into<Error>` for noproc and the
runtime being stopped because a number of functions exclusively error in
these cases, and especially the runtime being stopped has meaningful
implications for API, since it's not recoverable.

Signed-off-by: Nathan Perry <nathan@tailscale.com>
Change-Id: Ie78cc7b6b03fe5972ebb57ae04e66f5a6a6a6964
Signed-off-by: Nathan Perry <nathan@tailscale.com>
Change-Id: I6d4f0d152bb4d8c46fad0c6c9a97d4e66a6a6964
Signed-off-by: Nathan Perry <nathan@tailscale.com>
Change-Id: I28114c8c3342623a807154d688f5fec96a6a6964
@npry
npry merged commit 26db610 into main Nov 5, 2026
23 checks passed
@npry
npry deleted the npry/actor branch November 5, 2026 19:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants