Pykka API

pykka.__version__

Pykka’s PEP 386 and PEP 396 compatible version number

Actors

exception pykka.ActorDeadError

Exception raised when trying to use a dead or unavailable actor.

class pykka.Actor(*args, **kwargs)

To create an actor:

1. subclass one of the Actor implementations, e.g. GeventActor or ThreadingActor,
2. implement your methods, including __init__(), as usual,
3. call Actor.start() on your actor class, passing the method any arguments for your constructor.

To stop an actor, call Actor.stop() or ActorRef.stop().

For example:

import pykka

class MyActor(pykka.ThreadingActor):
    def __init__(self, my_arg=None):
        super(MyActor, self).__init__()
        ... # My optional init code with access to start() arguments

    def on_start(self):
        ... # My optional setup code in same context as on_receive()

    def on_stop(self):
        ... # My optional cleanup code in same context as on_receive()

    def on_failure(self, exception_type, exception_value, traceback):
        ... # My optional cleanup code in same context as on_receive()

    def on_receive(self, message):
        ... # My optional message handling code for a plain actor

    def a_method(self, ...):
        ... # My regular method to be used through an ActorProxy

my_actor_ref = MyActor.start(my_arg=...)
my_actor_ref.stop()

classmethod start(*args, **kwargs)

Start an actor and register it in the ActorRegistry.

Any arguments passed to start() will be passed on to the class constructor.

Behind the scenes, the following is happening when you call start():

1. The actor is created:
   1. actor_urn is initialized with the assigned URN.
   2. actor_inbox is initialized with a new actor inbox.
   3. actor_ref is initialized with a pykka.ActorRef object for safely communicating with the actor.
   4. At this point, your __init__() code can run.
2. The actor is registered in pykka.ActorRegistry.
3. The actor receive loop is started by the actor’s associated thread/greenlet.

Returns:a ActorRef which can be used to access the actor in a safe manner

actor_urn = None

The actor URN string is a universally unique identifier for the actor. It may be used for looking up a specific actor using ActorRegistry.get_by_urn.

actor_inbox = None

The actor’s inbox. Use ActorRef.tell(), ActorRef.ask(), and friends to put messages in the inbox.

actor_stopped = None

A threading.Event representing whether or not the actor should continue processing messages. Use stop() to change it.

actor_ref = None

The actor’s ActorRef instance.

stop()

Stop the actor.

It’s equivalent to calling ActorRef.stop() with block=False.

on_start()

Hook for doing any setup that should be done after the actor is started, but before it starts processing messages.

For ThreadingActor, this method is executed in the actor’s own thread, while __init__() is executed in the thread that created the actor.

If an exception is raised by this method the stack trace will be logged, and the actor will stop.

on_stop()

Hook for doing any cleanup that should be done after the actor has processed the last message, and before the actor stops.

This hook is not called when the actor stops because of an unhandled exception. In that case, the on_failure() hook is called instead.

For ThreadingActor this method is executed in the actor’s own thread, immediately before the thread exits.

If an exception is raised by this method the stack trace will be logged, and the actor will stop.

on_failure(exception_type, exception_value, traceback)

Hook for doing any cleanup after an unhandled exception is raised, and before the actor stops.

For ThreadingActor this method is executed in the actor’s own thread, immediately before the thread exits.

The method’s arguments are the relevant information from sys.exc_info().

If an exception is raised by this method the stack trace will be logged, and the actor will stop.

on_receive(message)

May be implemented for the actor to handle regular non-proxy messages.

Messages where the value of the “command” key matches “pykka_*” are reserved for internal use in Pykka.

Parameters:message (picklable dict) – the message to handle Returns:anything that should be sent as a reply to the sender

class pykka.ThreadingActor(*args, **kwargs)

ThreadingActor implements Actor using regular Python threads.

This implementation is slower than GeventActor, but can be used in a process with other threads that are not Pykka actors.

use_daemon_thread = False

A boolean value indicating whether this actor is executed on a thread that is a daemon thread (True) or not (False). This must be set before pykka.Actor.start() is called, otherwise RuntimeError is raised.

The entire Python program exits when no alive non-daemon threads are left. This means that an actor running on a daemon thread may be interrupted at any time, and there is no guarantee that cleanup will be done or that pykka.Actor.on_stop() will be called.

Actors do not inherit the daemon flag from the actor that made it. It always has to be set explicitly for the actor to run on a daemonic thread.

class pykka.ActorRef(actor)

Reference to a running actor which may safely be passed around.

ActorRef instances are returned by Actor.start() and the lookup methods in ActorRegistry. You should never need to create ActorRef instances yourself.

Parameters:actor (Actor) – the actor to wrap

actor_class = None

The class of the referenced actor.

actor_urn = None

See Actor.actor_urn.

actor_inbox = None

See Actor.actor_inbox.

actor_stopped = None

See Actor.actor_stopped.

is_alive()

Check if actor is alive.

This is based on the actor’s stopped flag. The actor is not guaranteed to be alive and responding even though is_alive() returns True.

Returns:Returns True if actor is alive, False otherwise.

tell(message)

Send message to actor without waiting for any response.

Will generally not block, but if the underlying queue is full it will block until a free slot is available.

Parameters:message (picklable dict) – message to send Raise:pykka.ActorDeadError if actor is not available Returns:nothing

ask(message, block=True, timeout=None)

Send message to actor and wait for the reply.

The message must be a picklable dict. If block is False, it will immediately return a Future instead of blocking.

If block is True, and timeout is None, as default, the method will block until it gets a reply, potentially forever. If timeout is an integer or float, the method will wait for a reply for timeout seconds, and then raise pykka.Timeout.

Parameters:

• message (picklable dict) – message to send
• block (boolean) – whether to block while waiting for a reply
• timeout (float or None) – seconds to wait before timeout if blocking

Raise:

pykka.Timeout if timeout is reached if blocking

Raise:

any exception returned by the receiving actor if blocking

Returns:

pykka.Future, or response if blocking

stop(block=True, timeout=None)

Send a message to the actor, asking it to stop.

Returns True if actor is stopped or was being stopped at the time of the call. False if actor was already dead. If block is False, it returns a future wrapping the result.

Messages sent to the actor before the actor is asked to stop will be processed normally before it stops.

Messages sent to the actor after the actor is asked to stop will be replied to with pykka.ActorDeadError after it stops.

The actor may not be restarted.

block and timeout works as for ask().

Returns:pykka.Future, or a boolean result if blocking

proxy()

Wraps the ActorRef in an ActorProxy.

Using this method like this:

proxy = AnActor.start().proxy()

is analogous to:

proxy = ActorProxy(AnActor.start())

Raise:pykka.ActorDeadError if actor is not available Returns:pykka.ActorProxy

Proxies

class pykka.ActorProxy(actor_ref, attr_path=None)

An ActorProxy wraps an ActorRef instance. The proxy allows the referenced actor to be used through regular method calls and field access.

You can create an ActorProxy from any ActorRef:

actor_ref = MyActor.start()
actor_proxy = ActorProxy(actor_ref)

You can also get an ActorProxy by using proxy():

actor_proxy = MyActor.start().proxy()

When reading an attribute or getting a return value from a method, you get a Future object back. To get the enclosed value from the future, you must call get() on the returned future:

print actor_proxy.string_attribute.get()
print actor_proxy.count().get() + 1

If you call a method just for it’s side effects and do not care about the return value, you do not need to accept the returned future or call get() on the future. Simply call the method, and it will be executed concurrently with your own code:

actor_proxy.method_with_side_effect()

If you want to block your own code from continuing while the other method is processing, you can use get() to block until it completes:

actor_proxy.method_with_side_effect().get()

An actor can use a proxy to itself to schedule work for itself. The scheduled work will only be done after the current message and all messages already in the inbox are processed.

For example, if an actor can split a time consuming task into multiple parts, and after completing each part can ask itself to start on the next part using proxied calls or messages to itself, it can react faster to other incoming messages as they will be interleaved with the parts of the time consuming task. This is especially useful for being able to stop the actor in the middle of a time consuming task.

To create a proxy to yourself, use the actor’s actor_ref attribute:

proxy_to_myself_in_the_future = self.actor_ref.proxy()

If you create a proxy in your actor’s constructor or on_start method, you can create a nice API for deferring work to yourself in the future:

def __init__(self):
    ...
    self.in_future = self.actor_ref.proxy()
    ...

def do_work(self):
    ...
    self.in_future.do_more_work()
    ...

def do_more_work(self):
    ...

An example of ActorProxy usage:

#! /usr/bin/env python

import pykka


class Adder(pykka.ThreadingActor):

    def add_one(self, i):
        print('{} is increasing {}'.format(self, i))
        return i + 1


class Bookkeeper(pykka.ThreadingActor):

    def __init__(self, adder):
        super(Bookkeeper, self).__init__()
        self.adder = adder

    def count_to(self, target):
        i = 0
        while i < target:
            i = self.adder.add_one(i).get()
            print('{} got {} back'.format(self, i))


if __name__ == '__main__':
    adder = Adder.start().proxy()
    bookkeeper = Bookkeeper.start(adder).proxy()
    bookkeeper.count_to(10).get()
    pykka.ActorRegistry.stop_all()

Parameters:actor_ref (pykka.ActorRef) – reference to the actor to proxy Raise:pykka.ActorDeadError if actor is not available

actor_ref = None

The actor’s pykka.ActorRef instance.

Futures

exception pykka.Timeout

Exception raised at future timeout.

class pykka.Future

A Future is a handle to a value which are available or will be available in the future.

Typically returned by calls to actor methods or accesses to actor fields.

To get hold of the encapsulated value, call Future.get().

get(timeout=None)

Get the value encapsulated by the future.

If the encapsulated value is an exception, it is raised instead of returned.

If timeout is None, as default, the method will block until it gets a reply, potentially forever. If timeout is an integer or float, the method will wait for a reply for timeout seconds, and then raise pykka.Timeout.

The encapsulated value can be retrieved multiple times. The future will only block the first time the value is accessed.

Parameters:timeout (float or None) – seconds to wait before timeout Raise:pykka.Timeout if timeout is reached Raise:encapsulated value if it is an exception Returns:encapsulated value if it is not an exception

set(value=None)

Set the encapsulated value.

Parameters:value (any picklable object or None) – the encapsulated value or nothing Raise:an exception if set is called multiple times

set_exception(exc_info=None)

Set an exception as the encapsulated value.

You can pass an exc_info three-tuple, as returned by sys.exc_info(). If you don’t pass exc_info, sys.exc_info() will be called and the value returned by it used.

In other words, if you’re calling set_exception(), without any arguments, from an except block, the exception you’re currently handling will automatically be set on the future.

Changed in version 0.15: Previously, set_exception() accepted an exception instance as its only argument. This still works, but it is deprecated and will be removed in a future release.

Parameters:exc_info (three-tuple of (exc_class, exc_instance, traceback)) – the encapsulated exception

set_get_hook(func)

Set a function to be executed when get() is called.

The function will be called when get() is called, with the timeout value as the only argument. The function’s return value will be returned from get().

New in version 1.2.

Parameters:func (function accepting a timeout value) – called to produce return value of get()

filter(func)

Return a new future with only the items passing the predicate function.

If the future’s value is an iterable, filter() will return a new future whose value is another iterable with only the items from the first iterable for which func(item) is true. If the future’s value isn’t an iterable, a TypeError will be raised when get() is called.

Example:

>>> import pykka
>>> f = pykka.ThreadingFuture()
>>> g = f.filter(lambda x: x > 10)
>>> g
<pykka.future.ThreadingFuture at ...>
>>> f.set(range(5, 15))
>>> f.get()
[5, 6, 7, 8, 9, 10, 11, 12, 13, 14]
>>> g.get()
[11, 12, 13, 14]

New in version 1.2.

join(*futures)

Return a new future with a list of the result of multiple futures.

One or more futures can be passed as arguments to join(). The new future returns a list with the results from all the joined futures.

Example:

>>> import pykka
>>> a = pykka.ThreadingFuture()
>>> b = pykka.ThreadingFuture()
>>> c = pykka.ThreadingFuture()
>>> f = a.join(b, c)
>>> a.set('def')
>>> b.set(123)
>>> c.set(False)
>>> f.get()
['def', 123, False]

New in version 1.2.

map(func)

Return a new future with the result of the future passed through a function.

If the future’s result is a single value, it is simply passed to the function. If the future’s result is an iterable, the function is applied to each item in the iterable.

Example:

>>> import pykka
>>> f = pykka.ThreadingFuture()
>>> g = f.map(lambda x: x + 10)
>>> f.set(30)
>>> g.get()
40

>>> f = pykka.ThreadingFuture()
>>> g = f.map(lambda x: x + 10)
>>> f.set([30, 300, 3000])
>>> g.get()
[40, 310, 3010]

New in version 1.2.

reduce(func[, initial])

Return a new future with the result of reducing the future’s iterable into a single value.

The function of two arguments is applied cumulatively to the items of the iterable, from left to right. The result of the first function call is used as the first argument to the second function call, and so on, until the end of the iterable. If the future’s value isn’t an iterable, a TypeError is raised.

reduce() accepts an optional second argument, which will be used as an initial value in the first function call. If the iterable is empty, the initial value is returned.

Example:

>>> import pykka
>>> f = pykka.ThreadingFuture()
>>> g = f.reduce(lambda x, y: x + y)
>>> f.set(['a', 'b', 'c'])
>>> g.get()
'abc'

>>> f = pykka.ThreadingFuture()
>>> g = f.reduce(lambda x, y: x + y)
>>> f.set([1, 2, 3])
>>> (1 + 2) + 3
6
>>> g.get()
6

>>> f = pykka.ThreadingFuture()
>>> g = f.reduce(lambda x, y: x + y, 5)
>>> f.set([1, 2, 3])
>>> ((5 + 1) + 2) + 3
11
>>> g.get()
11

>>> f = pykka.ThreadingFuture()
>>> g = f.reduce(lambda x, y: x + y, 5)
>>> f.set([])
>>> g.get()
5

New in version 1.2.

class pykka.ThreadingFuture

ThreadingFuture implements Future for use with ThreadingActor.

The future is implemented using a Queue.Queue.

The future does not make a copy of the object which is set() on it. It is the setters responsibility to only pass immutable objects or make a copy of the object before setting it on the future.

Changed in version 0.14: Previously, the encapsulated value was a copy made with copy.deepcopy(), unless the encapsulated value was a future, in which case the original future was encapsulated.

get(timeout=None)

Get the value encapsulated by the future.

If the encapsulated value is an exception, it is raised instead of returned.

If timeout is None, as default, the method will block until it gets a reply, potentially forever. If timeout is an integer or float, the method will wait for a reply for timeout seconds, and then raise pykka.Timeout.

The encapsulated value can be retrieved multiple times. The future will only block the first time the value is accessed.

Parameters:timeout (float or None) – seconds to wait before timeout Raise:pykka.Timeout if timeout is reached Raise:encapsulated value if it is an exception Returns:encapsulated value if it is not an exception

set(value=None)

Set the encapsulated value.

Parameters:value (any picklable object or None) – the encapsulated value or nothing Raise:an exception if set is called multiple times

set_exception(exc_info=None)

Set an exception as the encapsulated value.

You can pass an exc_info three-tuple, as returned by sys.exc_info(). If you don’t pass exc_info, sys.exc_info() will be called and the value returned by it used.

In other words, if you’re calling set_exception(), without any arguments, from an except block, the exception you’re currently handling will automatically be set on the future.

Changed in version 0.15: Previously, set_exception() accepted an exception instance as its only argument. This still works, but it is deprecated and will be removed in a future release.

Parameters:exc_info (three-tuple of (exc_class, exc_instance, traceback)) – the encapsulated exception

pykka.get_all(futures, timeout=None)

Collect all values encapsulated in the list of futures.

If timeout is not None, the method will wait for a reply for timeout seconds, and then raise pykka.Timeout.

Parameters:

• futures (list of pykka.Future) – futures for the results to collect
• timeout (float or None) – seconds to wait before timeout

Raise:

pykka.Timeout if timeout is reached

Returns:

list of results

Registry

class pykka.ActorRegistry

Registry which provides easy access to all running actors.

Contains global state, but should be thread-safe.

classmethod broadcast(message, target_class=None)

Broadcast message to all actors of the specified target_class.

If no target_class is specified, the message is broadcasted to all actors.

Parameters:

• message (picklable dict) – the message to send
• target_class (class or class name) – optional actor class to broadcast the message to

classmethod get_all()

Get ActorRef for all running actors.

Returns:list of pykka.ActorRef

classmethod get_by_class(actor_class)

Get ActorRef for all running actors of the given class, or of any subclass of the given class.

Parameters:actor_class (class) – actor class, or any superclass of the actor Returns:list of pykka.ActorRef

classmethod get_by_class_name(actor_class_name)

Get ActorRef for all running actors of the given class name.

Parameters:actor_class_name (string) – actor class name Returns:list of pykka.ActorRef

classmethod get_by_urn(actor_urn)

Get an actor by its universally unique URN.

Parameters:actor_urn (string) – actor URN Returns:pykka.ActorRef or None if not found

classmethod register(actor_ref)

Register an ActorRef in the registry.

This is done automatically when an actor is started, e.g. by calling Actor.start().

Parameters:actor_ref (pykka.ActorRef) – reference to the actor to register

classmethod stop_all(block=True, timeout=None)

Stop all running actors.

block and timeout works as for ActorRef.stop().

If block is True, the actors are guaranteed to be stopped in the reverse of the order they were started in. This is helpful if you have simple dependencies in between your actors, where it is sufficient to shut down actors in a LIFO manner: last started, first stopped.

If you have more complex dependencies in between your actors, you should take care to shut them down in the required order yourself, e.g. by stopping dependees from a dependency’s on_stop() method.

Returns:If not blocking, a list with a future for each stop action. If blocking, a list of return values from pykka.ActorRef.stop().

classmethod unregister(actor_ref)

Remove an ActorRef from the registry.

This is done automatically when an actor is stopped, e.g. by calling Actor.stop().

Parameters:actor_ref (pykka.ActorRef) – reference to the actor to unregister

Gevent support

class pykka.gevent.GeventFuture(async_result=None)

GeventFuture implements pykka.Future for use with GeventActor.

It encapsulates a gevent.event.AsyncResult object which may be used directly, though it will couple your code with gevent.

async_result = None

The encapsulated gevent.event.AsyncResult

get(timeout=None)

Get the value encapsulated by the future.

If the encapsulated value is an exception, it is raised instead of returned.

If timeout is None, as default, the method will block until it gets a reply, potentially forever. If timeout is an integer or float, the method will wait for a reply for timeout seconds, and then raise pykka.Timeout.

The encapsulated value can be retrieved multiple times. The future will only block the first time the value is accessed.

Parameters:timeout (float or None) – seconds to wait before timeout Raise:pykka.Timeout if timeout is reached Raise:encapsulated value if it is an exception Returns:encapsulated value if it is not an exception

set(value=None)

Set the encapsulated value.

Parameters:value (any picklable object or None) – the encapsulated value or nothing Raise:an exception if set is called multiple times

set_exception(exc_info=None)

Set an exception as the encapsulated value.

You can pass an exc_info three-tuple, as returned by sys.exc_info(). If you don’t pass exc_info, sys.exc_info() will be called and the value returned by it used.

In other words, if you’re calling set_exception(), without any arguments, from an except block, the exception you’re currently handling will automatically be set on the future.

Changed in version 0.15: Previously, set_exception() accepted an exception instance as its only argument. This still works, but it is deprecated and will be removed in a future release.

Parameters:exc_info (three-tuple of (exc_class, exc_instance, traceback)) – the encapsulated exception

class pykka.gevent.GeventActor(*args, **kwargs)

GeventActor implements pykka.Actor using the gevent library. gevent is a coroutine-based Python networking library that uses greenlet to provide a high-level synchronous API on top of libevent event loop.

This is a very fast implementation, but as of gevent 0.13.x it does not work in combination with other threads.

Eventlet support

class pykka.eventlet.EventletEvent

EventletEvent adapts eventlet.event.Event to threading.Event interface.

class pykka.eventlet.EventletFuture

EventletFuture implements pykka.Future for use with EventletActor.

get(timeout=None)

Get the value encapsulated by the future.

If the encapsulated value is an exception, it is raised instead of returned.

If timeout is None, as default, the method will block until it gets a reply, potentially forever. If timeout is an integer or float, the method will wait for a reply for timeout seconds, and then raise pykka.Timeout.

The encapsulated value can be retrieved multiple times. The future will only block the first time the value is accessed.

Parameters:timeout (float or None) – seconds to wait before timeout Raise:pykka.Timeout if timeout is reached Raise:encapsulated value if it is an exception Returns:encapsulated value if it is not an exception

set(value=None)

Set the encapsulated value.

Parameters:value (any picklable object or None) – the encapsulated value or nothing Raise:an exception if set is called multiple times

set_exception(exc_info=None)

Set an exception as the encapsulated value.

You can pass an exc_info three-tuple, as returned by sys.exc_info(). If you don’t pass exc_info, sys.exc_info() will be called and the value returned by it used.

In other words, if you’re calling set_exception(), without any arguments, from an except block, the exception you’re currently handling will automatically be set on the future.

Changed in version 0.15: Previously, set_exception() accepted an exception instance as its only argument. This still works, but it is deprecated and will be removed in a future release.

Parameters:exc_info (three-tuple of (exc_class, exc_instance, traceback)) – the encapsulated exception

class pykka.eventlet.EventletActor(*args, **kwargs)

EventletActor implements pykka.Actor using the eventlet library.

This implementation uses eventlet green threads.

Logging

Pykka uses Python’s standard logging module for logging debug statements and any unhandled exceptions in the actors. All log records emitted by Pykka are issued to the logger named “pykka”, or a sublogger of it.

Out of the box, Pykka is set up with logging.NullHandler as the only log record handler. This is the recommended approach for logging in libraries, so that the application developer using the library will have full control over how the log records from the library will be exposed to the application’s users. In other words, if you want to see the log records from Pykka anywhere, you need to add a useful handler to the root logger or the logger named “pykka” to get any log output from Pykka. The defaults provided by logging.basicConfig() is enough to get debug log statements out of Pykka:

import logging
logging.basicConfig(level=logging.DEBUG)

If your application is already using logging, and you want debug log output from your own application, but not from Pykka, you can ignore debug log messages from Pykka by increasing the threshold on the Pykka logger to “info” level or higher:

import logging
logging.basicConfig(level=logging.DEBUG)
logging.getLogger('pykka').setLevel(logging.INFO)

For more details on how to use logging, please refer to the Python standard library documentation.

Debug helpers

pykka.debug.log_thread_tracebacks(*args, **kwargs)

Logs at CRITICAL level a traceback for each running thread.

This can be a convenient tool for debugging deadlocks.

The function accepts any arguments so that it can easily be used as e.g. a signal handler, but it does not use the arguments for anything.

To use this function as a signal handler, setup logging with a logging.CRITICAL threshold or lower and make your main thread register this with the signal module:

import logging
import signal

import pykka.debug

logging.basicConfig(level=logging.DEBUG)
signal.signal(signal.SIGUSR1, pykka.debug.log_thread_tracebacks)

If your application deadlocks, send the SIGUSR1 signal to the process:

kill -SIGUSR1 <pid of your process>

Signal handler caveats:

• The function must be registered as a signal handler by your main thread. If not, signal.signal() will raise a ValueError.
• All signals in Python are handled by the main thread. Thus, the signal will only be handled, and the tracebacks logged, if your main thread is available to do some work. Making your main thread idle using time.sleep() is OK. The signal will awaken your main thread. Blocking your main thread on e.g. Queue.Queue.get() or pykka.Future.get() will break signal handling, and thus you won’t be able to signal your process to print the thread tracebacks.

The morale is: setup signals using your main thread, start your actors, then let your main thread relax for the rest of your application’s life cycle.

For a complete example of how to use this, see examples/deadlock_debugging.py in Pykka’s source code.

New in version 1.1.