Engineering

A look at how we build software that holds up under real data and real scale, told through work we've done.

Find the connection.

Pick any two items and watch the path between them light up. We built a graph visualization tool, and the query layer behind it, to surface hidden links between people, documents, places and accounts.

Explore our software development services

Connected in 4 steps: Report → Vendor → Office → Site B → Warehouse

Data in motion.

Documents, images and audio, by the terabyte. Our pipelines take it all in, read it, enrich it and index it, so a single search finds what matters in any format or language.

  1. Take inAny format, gigabytes to terabytes
  2. ReadDetect each file and extract its content
  3. EnrichAdd metadata, entities and classification
  4. IndexOne search across all of it

We build the options, then choose.

When the right answer isn't obvious, we don't argue about it. We build the contenders, test them on real data and ship the winner.

Merging records at scale

Merging records across a huge collection was too slow. So we built the staging index four ways and ran them head to head, round after round.

  • Elasticsearch
  • BoltDB
  • BuntDB
  • In memoryChosen

In memory won on speed, so that is what shipped.

Moving data through a pipeline

A high-volume ingest had to scale out, fast. We put the main ways of passing work between parts of a system to the test.

  • Worker threads
  • Message queue (RabbitMQ)
  • Event stream (Kafka)
  • In-memory message busChosen

Inside a single service, the in-memory bus won, moving tens of thousands of messages a second. Where messages must survive a restart, we use persistent queues.

Choosing a front-end framework

In 2017 we built the same pilot three times, once in each framework, before we committed.

  • Angular
  • React
  • VueChosen

We picked Vue when it was the least popular of the three.

More from our work

Security authorization, start to finish

Our cybersecurity engineers have taken multiple systems all the way to an Authority to Operate, handling the evidence, documentation, scans, fixes and control gaps along the way.

AI built into the product

In a product we support, AI works where the work happens. It helps generate what is needed to bring in new data, instead of waiting in a chat window.

Find out what we can do
for you today.