Market Simulation

Perspective is a powerful data visualization and exploration tool for the browser. However, the types of applications that it was originally designed for, applications like interactive real-time trade blotters, may require complex and pricey data feeds. When Perspective was open-sourced, replicating these rapidly updating time-series visualizations with freely-accessible data was a challenge, making it difficult to create examples for the project. After trying various free-tier data feeds from crypto & conventional market data providers, we ultimately decided to generate our own fake market data in the browser, decoupling our examples from potentially-spotty web services and allowing us to fine-tune the data to show off what Perspective can really do. The market data example was born!

In this example, we've created a simulation of a simple market for a single security. It uses a Perspective Table to accumulate orders, and View queries to update the market state on a loop. Random orders are generated in JavaScript with a simple normal distribution around the market conditions queried from the Table, such as best open price.

Each visualization on this page is a <perspective-viewer> UI component bound to this Table, each with it's own distinct query giving a unique view on the same market data. This simulation is not designed to be an accurate market model, but rather to generate randomized data that looks like the distribution of real financial data.

What is a blotter

A blotter is a time-series of market data. Each row represents a limit order in our fake market, e.g. an offer to buy or sell up to a specified price. The raw data looks like this.

Most of these column values are randomly chosen once per row, then once written never change. The side columns describes whether the order is a "buy" or "sell" order, which determines how we should regard the price column, e.g. what limit price the order will buy or sell to.

Only the status column is updated in-place in the Perspective Table as the market simulation proceeds, starting in the "open" state, then proceeding to "closed" when the order is filled (more on that later), or "expired" if the order goes unfilled for some time.

Here's the same blotter with these columns visualized. Try executing some trades by clicking and holding the "Buy" button in the header to see how orders are inserted in real-time.

Most of these column values are randomly chosen once per row, then once written never change. The side columns describes whether the order is a "buy" or "sell" order, which determines how we should regard the price column, e.g. what limit price the order will buy or sell to.

Only the status column is updated in-place in the Perspective Table as the market simulation proceeds, starting in the "open" state, then proceeding to "closed" when the order is filled (more on that later), or "expired" if the order goes unfilled for some time.

You can use Perspective's color field on an X/Y Scatter chart to clearly see the status column's behavior. In this chart, every order's price is plotted, and it's color is determined by an expression column defined as side (if it's state is "open"), or just state otherwise (when it is "closed" or "expired"), yielding the possible values "buy", "sell", "closed", "expired". On the right, newly-entered trades appear as open, but are quickly closed (in 10s of simulation time) if the order goes unfilled.

Simulation

The market data is inserted into a Perspective Table, where it accumulates. This Table powers both the internal market simulation which updates orders and sets the market price, but also the `View` wueries that power all of the visualizations on this page.

The simulation proceeds in steps:

1. Insert a batch of random orders in a range around the market's bid/ask spread, the highest open bid and the lowest open ask. For each side of the order book, we can calculate the best open price by querying the orders Table using the "max" or "min" aggregates (respectively).

{
    "columns": ["price"],
    "group_by": ["security"],
    "aggregates": { "price": "max" },
    "filter": [
        ["side", "==", "buy"],
        ["status", "==", "open"]
    ]
}

2. Clear any matched orders in the Table. To match orders, we fetch all open orders on both sides which are outside of the best price, then update an equal number of both "buy" and "sell" side orders to "closed". The sort field guarantees that orders are closed in best, then oldest, order.

{
    columns: ["id"],
    filter: [
        ["side", "==", "buy"],
        ["status", "==", "open"],
        ["price", ">", price],
    ],
    sort: [
        ["price", "desc"],
        ["timestamp", "asc"],
    ],
}

3. Expire any elapsed orders which are still "open" by fetching orders older than the expiration ID and update thair status' to "expired".

{
    "columns": ["id"],
    "filter": [
        ["status", "==", "open"],
        ["id", "<", 12345]
    ]
}

4. Sleep for a bit and repeat (1).

When all is said and done, the order accumulation history looks like this, using Perspective's bucket() expression column to aggregate orders by price level (nearest dollar). You can see the "buy" and "sell" sides of the order book accumulating either "closed" or "expired" trades around the mid prices as simulation time progresses.

If we turn this into a Datagrid and use Perspective's split_by feature to create a separate column group per "side" and change the aggregate type to "count", we get an convention visualization for depth of book, e.g. how much open order volumes exists on what price levels for both trade sides.

The difference between the lowest "sell" order and the highest "buy" order is called the "spread". If we plot these values for all "closed" orders, we can see the price history (here bucketted in 15 second buckets for visibility).

Combining all of these visualizations together - we can bin the Y-axis by "price" level (to the nearest dollar), and the X-axis some time bucket (1 simulation minute in this example), we finally come to the visualization in the article heard, the order book's depth over time, with color indicating the order "side". Try changing these values in Perspective's ExprTK editor by clicking on the column settings button!