System Design FAQ: Top Questions
31. How would you design a Secret Management System?
A Secret Management System securely stores, rotates, and provides access to sensitive credentials such as API keys, database passwords, TLS certs, and tokens. It ensures security, compliance, and minimal access.
๐ Functional Requirements
- Securely store encrypted secrets at rest
- Access control and audit logging
- Auto-expiration and rotation of secrets
- Support for secrets versioning and revocation
๐ฆ Non-Functional Requirements
- Low-latency retrieval
- High durability and fault-tolerance
- Integration with CI/CD pipelines and cloud workloads
๐๏ธ Core Components
- Secrets Store: Encrypted KV store (Vault, AWS Secrets Manager)
- Access Policy Engine: RBAC/ABAC enforcement
- Audit Logger: Tracks access attempts and mutations
- Rotation Controller: Auto-renews secrets before expiry
๐ Example: HashiCorp Vault Secret Storage
# Enable KV v2 secret engine
vault secrets enable -path=secret kv-v2
# Store a secret
vault kv put secret/db password="hunter2"
# Retrieve a secret
vault kv get secret/db
๐ Vault Policy Example (HCL)
path "secret/data/db" {
capabilities = ["read"]
}
path "secret/data/db" {
capabilities = ["update"]
}
๐ Secret Rotation Strategy
- Configure TTL and rotation
- Integrate with external rotation scripts (e.g., DB user password reset)
- Emit rotation success/failure metrics
๐งช Secret Access with AWS SDK (Python)
import boto3
import base64
from botocore.exceptions import ClientError
def get_secret(secret_name):
client = boto3.client('secretsmanager')
try:
response = client.get_secret_value(SecretId=secret_name)
if 'SecretString' in response:
return response['SecretString']
else:
return base64.b64decode(response['SecretBinary'])
except ClientError as e:
print(f"Error retrieving secret: {e}")
return None
๐ Observability
- Secrets access frequency per user
- Unauthorized access attempts
- Rotation success/failure metrics
๐งฐ Tools/Infra Used
- Vault / AWS Secrets Manager / GCP Secret Manager
- KMS: AWS KMS, HashiCorp Transit, Google KMS
- Logging: Loki / ELK
๐ Final Insight
Secret management systems are essential for modern applications and cloud environments. Design choices must emphasize security, auditability, and seamless integration with tools, workflows, and automation.
