How Unwritten powers large-scale data processing with zero infrastructure overhead
Using Shuttle's Rust deployment platform to accelerate large-scale data processing without infrastructure complexity
- Up to 3x faster data processingPolars running natively in Rust delivers up to 3× faster performance than its Python counterpart
- Days to minutes deploymentInfrastructure setup reduced from weeks to minutes with zero specialised expertise
- Team focused on core expertiseDevelopment team concentrates on data pipelines instead of infrastructure management
The Challenge: Infrastructure Complexity Blocking Core Development
Companies and investors face more pronounced climate risks than ever before. Unwritten needed to create a single component capable of both performing CRUD operations on user-entered data and executing data processing workflows. The platform must serve enterprise clients who require strict security and often demand individual deployments.
Their previous approach of separating these workflows introduced unnecessary complexity and latency. For a team focused on large-scale data processing, infrastructure management was a distraction from their core expertise.
"Our core expertise and value lie in climate risk modelling, so we wanted to focus on optimising data pipelines rather than troubleshooting flaky web servers," explains the development team.
The challenge was compounded by their technology choices. For this specific greenfield project, they decided to adopt Rust as "an entirely new language" to leverage Polars for fast data transformations and Axum as a type-safe server framework. The team chose Polars running natively in Rust, as it delivered up to 3× faster performance than its Python counterpart, offering superior parallel computing capabilities that scale efficiently with growing data volumes.
However, setting up production-ready, scalable infrastructure for Rust applications traditionally required significant DevOps expertise.
Choosing Shuttle to Eliminate Infrastructure Overhead
Since Rust was their primary language choice for this greenfield component, Unwritten didn't directly compare Shuttle with other platforms. Instead, they used their experience with Vercel for front-end deployments as a benchmark when assessing Shuttle's features. Shuttle's position as a hosting platform specifically designed for Rust made it the obvious choice, with its approach of eliminating deployment complexity crucial for adopting a new language.
The integration spans their entire development lifecycle:
Given Rust's steep learning curve for the team adopting this entirely new language, offloading deployment complexity allowed them to focus on learning Rust and concentrate on development rather than infrastructure concerns.
"What took days to weeks to set up with traditional infrastructure now takes minutes with Shuttle. It's like Vercel for backends."Phillip Marks, CRO & CTO Unwritten
Focusing on Data Processing Over Infrastructure Management
Since adopting Shuttle, Unwritten has transformed how they approach infrastructure and development:
No more Terraform configurations or build scripts. The team no longer needs expertise in DevOps, cloud infrastructure, or networking.
Accelerated Development Cycles
Fast-paced development with frequent pipeline improvements driven by weekly customer feedback, with reliable rollbacks.
"Shuttle allows us to focus on what we do best" summarizes the team's experience. With Shuttle, they can deploy production-ready systems in minutes.
"Shuttle allows us to focus on what we do best" - Unwritten team
Technical Implementation
Unwritten leverages Shuttle's ecosystem for their data processing platform:
- 01Shuttlefor deployment and infrastructure management
- 02Axumfor type-safe REST API endpoints
- 03MongoDBfor database
- 04Polarsfor high-performance data transformations
- 05Dagsterfor data orchestration and ETL workflows
- 06Databricksfor heavy computational workloads
- 07PropelAuthfor authentication
- 08Better Stackfor monitoring and observability
- 09GitHub Actionsfor CI/CD automation
The combination provides stability and type safety paramount for data calculations where accuracy is critical. The team values Rust's compile-time guarantees and type safety as essential safeguards for their data pipelines, ensuring production-grade reliability standards.
Impact on Team and Workflow
Shuttle has unified the development process, making deployment experiences consistent across front-end and back-end development. This is particularly valuable for Unwritten's specialised team who excel in data engineering but were new to Rust development and prefer to avoid infrastructure complexity.
"From experience on previous projects with IaaS in public cloud environments, it took ordinary developers days to weeks to set up production-ready infrastructure," the team notes.
With Shuttle, infrastructure concerns disappear—you can have a fully operational setup in minutes with zero specialised expertise. The team describes Shuttle as "Vercel for backend."
Unwritten's Transformation
Unwritten's success demonstrates how Shuttle enables technology companies to focus on their core mission rather than infrastructure management. The combination of Rust's performance and Shuttle's deployment simplicity allowed their team to build production-ready data processing solutions at unprecedented speed.