Our work

Case Study

Cloud Secrets Management Solution

Implementation of HashiCorp Vault for a Top-3 US Bank

Customer Challenge

A top 3 US Bank asked Citihub Consulting to implement HashiCorp Vault to provide a centralized secrets management solution for their very  large-scale cloud environments.

Like all enterprises, the bank was concerned about:

  • The sprawl of secrets management with credentials potentially being stored in unsecured locations e.g. source code, config management, content management solutions or logs
  • Keeping pace with identity and authentication needs of highly ephemeral cloud and container environments as existing tooling was designed for static environments

How Citihub Consulting Helped

Citihub Consulting provided an agile team (product owner, technical architect, developer, site reliability engineer and release engineer) with strong enterprise security knowledge who worked alongside bank resource to architect, design, build and deploy the HashiCorp Vault solution.

The solution included:

  • HashiCorp Consul storage backend* and integrated HSMs. While Vault offers support for other storage options, Consul is highly scalable and fault tolerant. It does a good job securing data at rest, while Vault secures data in transit.  Underneath the hood, it uses RAFT & SERF protocols, which you’ll find in products such as Kubernetes and Kafka.
  • A custom Vault authentication plugin developed by Citihub Consulting to integrate with the client’s custom entitlements backend
  • Automation to configure and initialize Consul and Vault servers including operational scripts to simply common operational tasks (e.g. disaster recovery, rekey operations, proactive health monitors, consul snapshots, log rotation and more)
  • Client onboarding automation using Terraform for namespace management and policy deployment
  • Performed knowledge transfer sessions
  • Operational hand-over included a custom performance benchmarking application and automated canary testing
  • SRE staff trained and automation developed to proactively ensure health

* In future versions of HashiCorp Vault, a separate Consul specific cluster will no longer be required, which will make the installation and upkeep much easier and reduce the infrastructure footprint by at least 30%.

Client Benefits

  • HashiCorp Vault operational in two regions with HA and two DR regions supporting dev, UAT and production environments
  • Centralized secrets management solution, integrated with the client’s HSM solution, to reduce and prevent further sprawl of secrets (e.g. key/value, Azure, transit)
  • Simple, automated service for applications to programmatically consume secrets with full auditability
  • Secrets and application data securely encrypted at rest and in flight
  • Reduced risks through ephemeral credentials reduce risk
  • Ability, when needed, to authenticate and access different cloud services, systems and end points using trusted identities through extensive and extensible plug-in capabilities (e.g. to Azure, AWS, GCP and GitHub services)