About Secrets Management

In the era of distributed systems and microservices architecture, it is extremely important to securely manage secrets (e.g. passwords, API keys, certificates). Otherwise, you may face serious security risks. HashiCorp Vault was born to solve this problem, providing a centralized solution to manage, store, and control access to secrets.

What is HashiCorp Vault?

HashiCorp Vault is an open source tool designed to manage secrets and protect sensitive data. It provides a unified interface for accessing and managing secrets, allowing you to:

• Store secrets securely.
• Control access to secrets.
• Generate dynamic credentials as required.
• Encrypt sensitive data.
• Log all access to secrets for auditing.

Key Features of Vault

1. Secure Secret Storage

Vault encrypts secrets before storing them in backend storage. It supports many different types of backend storage, including:

• Consul
• etcd
• Amazon S3
• ...and more.

Vault uses strong encryption algorithms to protect secrets from unauthorized access. Learn more about symmetric and asymmetric encryption here .

2. Dynamic Credentials

Vault can generate dynamic credentials on demand for different services. Instead of using static credentials stored in code or configuration files, applications can ask Vault to generate temporary credentials to access resources. After use, these credentials are automatically revoked.

For example, you can configure Vault to create dynamic database credentials. When an application needs to access the database, it asks Vault to create a temporary user and password. After the application completes, Vault automatically revokes these credentials, reducing the risk of attack.

3. Encryption as a Service

Vault provides the ability to encrypt data "in transit" and "at rest". Applications can use Vault to encrypt data before storing it in a database or transmitting it over a network. Vault provides a simple API for encrypting and decrypting data, reducing the encryption burden on applications.

4. Audit Logging

Vault records all access to secrets in audit logs. Audit logs provide detailed information about who accessed which secrets, when, and from where. This information is useful for investigating security incidents and complying with regulations.

Audit logs can be stored to a variety of backends, including file, syslog, and cloud storage.

Vault Architecture

The Vault architecture consists of the following main components:

• API Server: Provides a RESTful interface to interact with Vault.
• Authentication Backends: Authenticate users and applications.
• Secret Engines: Manage and create secrets.
• Storage Backend: Stores Vault data, including secrets and configuration.
• Audit Backend: Stores audit logs.

Vault Overview Architecture

The figure above shows the overall architecture of Vault. Users and applications interact with Vault through the API Server. The API Server authenticates requests and authorizes access to secrets. Secrets are stored in the Storage Backend and all access operations are logged in the Audit Backend.

Using Vault in Practice

1. Install and configure Vault

You can install Vault on a variety of platforms, including Linux, macOS, and Windows. HashiCorp provides installation packages and detailed instructions on their website.

After installation, you need to configure Vault to use a storage backend and authentication backend. You also need to initialize Vault and unseal it before you can use it.

 # Khởi tạo Vault vault operator init # Unseal Vault vault operator unseal # Đăng nhập vào Vault vault login 2. Store secrets # Khởi tạo Vault vault operator init # Unseal Vault vault operator unseal # Đăng nhập vào Vault vault login

Once the Vault is initialized and unsealed, you can start storing secrets in the Vault. You can use the Vault CLI, API, or UI to store secrets.

 # Lưu trữ một secret bằng CLI vault kv put secret/myapp/database password=mypassword # Đọc secret vault kv get secret/myapp/database 3. Create dynamic database credentials # Lưu trữ một secret bằng CLI vault kv put secret/myapp/database password=mypassword # Đọc secret vault kv get secret/myapp/database

To create dynamic database credentials, you need to configure a database secret engine. You can then create roles to define the access rights of the generated credentials.

 # Kích hoạt database secret engine vault secrets enable database # Cấu hình kết nối đến database vault write database/config/myapp \ plugin_name=mysql-database-ha \ allowed_roles=readonly,readwrite \ connection_url="root:password@tcp(localhost:3306)/" \ username="root" \ password="password" # Tạo role vault write database/roles/readonly \ db_name=myapp \ creation_statements="CREATE USER '{{name}}'@'%' IDENTIFIED BY '{{password}}';" \ default_ttl="1h" \ max_ttl="24h" \ renew_statements="REVOKE ALL PRIVILEGES ON *.* FROM '{{name}}'@'%';" \ revocation_statements="DROP USER '{{name}}'@'%';" \ rollback_statements="DROP USER '{{name}}'@'%';" # Lấy dynamic credentials vault read database/creds/readonly 4. Integrate with Kubernetes # Kích hoạt database secret engine vault secrets enable database # Cấu hình kết nối đến database vault write database/config/myapp \ plugin_name=mysql-database-ha \ allowed_roles=readonly,readwrite \ connection_url="root:password@tcp(localhost:3306)/" \ username="root" \ password="password" # Tạo role vault write database/roles/readonly \ db_name=myapp \ creation_statements="CREATE USER '{{name}}'@'%' IDENTIFIED BY '{{password}}';" \ default_ttl="1h" \ max_ttl="24h" \ renew_statements="REVOKE ALL PRIVILEGES ON *.* FROM '{{name}}'@'%';" \ revocation_statements="DROP USER '{{name}}'@'%';" \ rollback_statements="DROP USER '{{name}}'@'%';" # Lấy dynamic credentials vault read database/creds/readonly

Vault can be integrated with Kubernetes to provision secrets to pods. You can use the Vault Agent Injector to automatically inject secrets into pods when they are created. To learn more about Kubernetes, you can refer to this article .

 apiVersion: apps/v1 kind: Deployment metadata: name: myapp spec: template: metadata: annotations: vault.hashicorp.com/agent-inject: 'true' vault.hashicorp.com/role: 'myapp' vault.hashicorp.com/agent-inject-secret-database: 'secret/myapp/database' spec: containers: - name: myapp image: myapp:latest env: - name: DB_PASSWORD valueFrom: secretKeyRef: name: database key: password 5. Integrate with CI/CD pipelines apiVersion: apps/v1 kind: Deployment metadata: name: myapp spec: template: metadata: annotations: vault.hashicorp.com/agent-inject: 'true' vault.hashicorp.com/role: 'myapp' vault.hashicorp.com/agent-inject-secret-database: 'secret/myapp/database' spec: containers: - name: myapp image: myapp:latest env: - name: DB_PASSWORD valueFrom: secretKeyRef: name: database key: password

Vault can be integrated into CI/CD pipelines to provide secrets for build and deployment. You can use the Vault CLI or API to retrieve secrets from Vault and use them in scripts.

 # Lấy secret từ Vault trong CI/CD pipeline PASSWORD=$(vault kv get -field=password secret/myapp/database) # Sử dụng secret trong script echo "Database password: $PASSWORD" Advantages of using Vault # Lấy secret từ Vault trong CI/CD pipeline PASSWORD=$(vault kv get -field=password secret/myapp/database) # Sử dụng secret trong script echo "Database password: $PASSWORD"

• Security: Vault encrypts secrets and controls access.
• Auditability: Vault logs all access activities to secrets.
• Centralized Management: Vault provides a centralized interface for managing secrets.
• Scalability: Vault can scale to meet the needs of large systems.
• Dynamic Secrets: Vault can create dynamic secrets on demand.

Disadvantages of Using Vault

• Complexity: Configuring and managing Vault can be complex.
• Cost: Vault requires resources to run and manage.
• Dependencies: Your system will depend on Vault to access secrets.

Frequently Asked Interview Questions on Secrets Management and Vault

1. How does Vault work? Explain the architecture of Vault.
2. What is the difference between static secrets and dynamic secrets? When should you use each?
3. How to integrate Vault with Kubernetes?
4. What authentication backends have you used in Vault? What are the pros and cons of each?
5. How to ensure high availability for Vault?
6. What problems have you encountered using Vault and how did you solve them?
7. What is the role of audit logs in Vault and how to use them effectively?

Conclude

HashiCorp Vault is a powerful tool for managing secrets in distributed systems. It provides security, auditability, and centralized management features, helping you protect sensitive data and comply with regulations. While Vault can be complex to configure and manage, the benefits are huge.

Hopefully this article has given you an overview of Secrets Management with Vault. Keep exploring and learning more to put Vault into practice and enhance the security of your system.