Implementing Workflows on App Engine with Fantasm

This post is another entry in our ongoing series of guest posts contributed by App Engine developers. Today we partner with Jason Collins and Shawn Rusaw of VendAsta Technologies who discuss their simplistic workflow management system for Python App Engine called Fantasm. Fantasm is very simple to start using but incredibly powerful. So powerful in fact, that they couldn't fit it into one blog post. So for more detail, see the Implementing Workflows on App Engine with Fantasm article in our documentation.

Most software systems of reasonable size need to implement workflows: a series of processing steps and decision points over an artifact, typically a document. You are likely using workflows even if you don't realize it; often processing is built up iteratively over time until you're left with a complex system with lots of subtleties. These systems can mysterious and difficult to manage or extend.

A formal workflow engine can offer help in these instances. It allows developers to break the steps and decision points into manageable and testable chunks, so that processing is predictable and measurable. Workflow engines can offer visibility into system operations and metrics around execution times and failure points. They provide retry mechanisms and allow tasks to be distributed among multiple computers.

We have developed Fantasm to be just such a workflow engine. Fantasm is a Python library that allows you to specify your tasks and steps between those tasks in a YAML configuration file. It uses a finite state machine model where the tasks are states and the steps are transitions. As a developer, you implement the code that executes inside of a state and are only responsible to return an event that is used to identify the appropriate transition to the next state. It is similar to the App Engine Pipeline API but is more simplistic and serves as a user-friendly introduction to the concept of workflows in App Engine apps.

Fantasm has become the central processing tool within our organization and we hope it can be of help to yours. For a more in-depth look at Fantasm (including examples!), please see the full Implementing Workflows on App Engine with Fantasm article in the App Engine documentation.

Posted by Wesley Chun, App Engine team