A look at how we build software that holds up under real data and real scale, told through work we've done.
Find the connection.
Pick any two items and watch the path between them light up. We built a graph visualization tool, and the query layer behind it, to surface hidden links between people, documents, places and accounts.
Connected in 4 steps: Report → Vendor → Office → Site B → Warehouse
Data in motion.
Documents, images and audio, by the terabyte. Our pipelines take it all in, read it, enrich it and index it, so a single search finds what matters in any format or language.
Take inAny format, gigabytes to terabytes
ReadDetect each file and extract its content
EnrichAdd metadata, entities and classification
IndexOne search across all of it
PDFJPGWAVDOCXPNGMP3TXT
bridge inspection report
Inspection report, Pier 4Document · matched on text
Photo of the north supportImage · matched by meaning
Informe de inspección del puenteSpanish · matched across languages
Field audio note, 02:14Audio · matched on the transcript
We build the options, then choose.
When the right answer isn't obvious, we don't argue about it. We build the contenders, test them on real data and ship the winner.
Merging records at scale
Merging records across a huge collection was too slow. So we built the staging index four ways and ran them head to head, round after round.
Elasticsearch
BoltDB
BuntDB
In memoryChosen
In memory won on speed, so that is what shipped.
Moving data through a pipeline
A high-volume ingest had to scale out, fast. We put the main ways of passing work between parts of a system to the test.
Worker threads
Message queue (RabbitMQ)
Event stream (Kafka)
In-memory message busChosen
Inside a single service, the in-memory bus won, moving tens of thousands of messages a second. Where messages must survive a restart, we use persistent queues.
Choosing a front-end framework
In 2017 we built the same pilot three times, once in each framework, before we committed.
Angular
React
VueChosen
We picked Vue when it was the least popular of the three.
More from our work
Security authorization, start to finish
Our cybersecurity engineers have taken multiple systems all the way to an Authority to Operate, handling the evidence, documentation, scans, fixes and control gaps along the way.
AI built into the product
In a product we support, AI works where the work happens. It helps generate what is needed to bring in new data, instead of waiting in a chat window.