Omlet House: Scaling Open Source Observability

Open-Source Observability stacks are always appealing. Alongside a core vendor, enterprises can realize the vision of a true “observability data lake” and leverage cutting-edge AI features, while also managing compliance constraints. Low-cost, highly customizable, and transparent, but not without challenges.

Configuration and Administration

Config and Admin are usually the first pain points engineers confront with OSS Observability solutions. Think endless YAML, user management (auth), service integration, boilerplate queries and dashboards, cluster configuration, permissions, etc.

Networking

Beyond initial configuration, networking components need to be addressed. How will users access the UIs? What DNS configurations do you need to think about? What about certificate management? Although these seem like simple, common-sense things, they really add up.

Scale

Different services and teams, at different times, may send unpredictable amounts of volume. How do you sustain peak load and not under-provision? How do you enforce resource quotas and minimize noisy neighbors? Can resources spin up (and down) quickly?

Multi-Tenancy and Orchestration

In the enterprise, various teams/groups will want their own tenants. How do you properly isolate for ownership and compliance? Can a central team manage and govern sub-units effectively?

Omlet House

The above factors, especially regarding enterprise multi-tenancy and orchestration, really pushed us to consider the ideal stack of open source observability components. There are various combinations one could think of, but the above challenges still remained. With our Omlet House service that enterprises can deploy in their own Kubernetes cluster, we wanted the “stack” spin-up to be fluid and ready to go.

If you are on AWS, GCP, or Azure, we’d love to talk to you about trying it out! Send us a note at info@omlet.co!