Password Vault - CLI Reference

Overview

FreeIPA vaults provide secure, encrypted storage for passwords, keys, certificates, and other sensitive secrets. Integrated with Dogtag’s Key Recovery Authority (KRA), vaults offer encrypted secret archival and retrieval with multiple encryption models to balance security and operational requirements. Each vault stores a single secret which can be archived and retrieved by authorized users, services, or groups based on vault membership and ownership.

Vaults address common secrets management challenges: eliminating unencrypted password storage in configuration files, enabling secure password sharing between team members, providing audit trails for secret access, and establishing escrow mechanisms for secret recovery. Secrets stored in vaults remain encrypted both in transit and at rest, with encryption keys managed by the KRA to prevent unauthorized access even by FreeIPA administrators.

The vault system operates on a container model where vaults are organized under vault containers associated with users, services, or shared access. Each vault within a container has an owner (who controls vault configuration) and members (who can archive and retrieve secrets). This access control model enables flexible secrets sharing while maintaining clear ownership and accountability.

Ownership Models

User Vaults

User vaults belong to specific IPA users and are stored in that user’s vault container. The owning user has full control over their vaults, can designate other users as owners or members, and determines the vault’s encryption type. User vaults support personal secrets management like storing personal passwords, SSH keys, or API tokens.

Private vaults are a special case of user vaults where the vault owner is the currently authenticated user. When commands omit the --user parameter, they operate on the current user’s vault container, creating or accessing private vaults.

User vault containers are created automatically when a user creates their first vault. Container ownership follows user ownership, enabling users to manage their own secret storage infrastructure.

Service Vaults

Service vaults belong to IPA service principals (HTTP/, LDAP/, etc.) and store secrets associated with services rather than users. Service vaults enable applications to securely store configuration secrets, database passwords, API keys, or encryption keys that services need for operation.

Service authentication to vaults uses service Kerberos credentials. Services must obtain service tickets to archive or retrieve secrets from their vaults. This ensures only the legitimate service (with valid Kerberos credentials) can access its secrets.

Service vaults support secrets sharing between services or granting service administrators access to service secrets. Service vault members can include users (for administrative access) or other services (for service-to-service secret sharing).

Shared Vaults

Shared vaults enable team-based secrets management where multiple users need access to common secrets. Shared vaults are owned by administrators but can have multiple users, groups, or services as members with archive/retrieve privileges.

Common shared vault use cases include team passwords (shared database credentials, service account passwords), shared certificates, shared encryption keys, or project-specific API tokens. Shared vaults provide controlled access to team secrets without distributing copies or storing secrets insecurely.

Shared vault containers require explicit creation and ownership assignment. Unlike user vault containers (created automatically), shared containers must be established before creating vaults within them.

Encryption Types

Standard Vaults

Standard vaults use transport encryption (TLS) to protect secrets in transit and KRA encryption for storage. Secrets are encrypted automatically by the KRA using its internal keys. Authorized vault members can retrieve secrets without providing additional credentials beyond their IPA authentication.

Standard vaults balance security and convenience: secrets remain encrypted at rest and in transit, but members don’t need to manage separate vault passwords or keys. This model suits secrets requiring access control but not additional encryption layers.

The KRA master key encrypts all standard vault secrets. While this provides strong encryption, it means KRA compromise (or administrator access to KRA keys) could theoretically decrypt vault contents. Organizations requiring defense against administrator access should use symmetric or asymmetric vaults.

Symmetric Vaults

Symmetric vaults require a password to encrypt secrets before archiving and decrypt secrets during retrieval. The vault password never leaves the client; secrets are encrypted client-side before transmission to the KRA. This ensures even KRA administrators cannot decrypt vault contents without the vault password.

Symmetric vault security depends entirely on password strength and protection. Lost passwords permanently render symmetric vault contents irretrievable (unless escrow is configured). Shared symmetric vaults require secure password distribution to all authorized members.

Vault passwords are distinct from user authentication passwords. The vault password specifically protects vault contents and can differ from the user’s IPA login password. This separation enables vault password changes without affecting user authentication.

Asymmetric Vaults

Asymmetric vaults use public key encryption, where secrets are encrypted with a public key during archival and decrypted with the corresponding private key during retrieval. The private key never leaves the client and is not stored in IPA, providing maximum security against server compromise.

Asymmetric vault workflow requires maintaining private keys securely. Public keys are stored with the vault, enabling anyone with vault membership to archive secrets, but only private key holders can retrieve them. This split enables scenarios where many people can contribute secrets but few can access them.

Lost private keys permanently render asymmetric vault contents irretrievable. Organizations using asymmetric vaults should establish private key backup and recovery procedures, potentially including key escrow to trusted administrators.

Vault Members and Owners

Owners

Vault owners control vault configuration including encryption type, vault password, public keys, and access control (adding/removing owners and members). Owners can modify vault metadata, delete vaults, and change encryption parameters. The creator of a vault automatically becomes an owner.

Multiple owners enable shared vault administration. Ownership can transfer to users, groups, or services. Group ownership enables team-based vault management where any team member can administer vaults.

Owner privileges include member privileges; owners can also archive and retrieve secrets in addition to managing vault configuration.

Members

Vault members can archive and retrieve secrets but cannot modify vault configuration, change access control, or delete vaults. Membership is the appropriate privilege level for users who need secret access without vault administration capabilities.

Members can include individual users, groups (enabling team access), or services (enabling automated secret access). Group membership is particularly powerful for team secrets, automatically granting access to all current group members.

Member privileges respect vault encryption types. Symmetric vault members must provide the vault password; asymmetric vault members must provide the private key. Standard vault members only need IPA authentication.

Secret Archival and Retrieval

Archiving a secret stores it in the vault, overwriting any previously archived secret. Vaults can only store one secret at a time; archiving a new secret replaces the old one. This single-secret model ensures clarity about current secret state but requires separate vaults for multiple secrets.

Secrets are arbitrary binary data with no size restrictions beyond practical limits. Common secret types include passwords (plaintext strings), private keys (PEM format), certificates, JSON configuration files, or encrypted data blobs.

Retrieval returns the currently archived secret to the client. For standard vaults, only IPA authentication is required. For symmetric vaults, the vault password must be provided. For asymmetric vaults, the private key must be provided. Retrieval operations are logged for audit purposes.

Best Practices

Use asymmetric vaults for maximum security: When secrets require protection even from FreeIPA administrators, use asymmetric encryption with well-protected private keys.

Leverage shared vaults for team secrets: Rather than sharing passwords insecurely (email, chat), store them in shared vaults with controlled membership.

Document vault purposes: Maintain clear records of what each vault stores and who should have access. This aids auditing and prevents orphaned secrets.

Backup vault passwords and keys: For symmetric and asymmetric vaults, establish secure backup procedures for passwords/keys. Lost credentials mean permanent data loss.

Use meaningful vault names: Name vaults descriptively (database_password, api_key_production) rather than generic names (secret1, vault_a).

Audit vault access regularly: Review vault membership and monitor retrieval logs to detect inappropriate access or unused vaults.

Rotate vault contents: Treat vaults like any other secrets management; rotate stored passwords/keys periodically and update vault contents.

Minimize vault membership: Grant access only to users/services truly requiring it. Overly broad membership weakens secrets protection.

Consider escrow for critical vaults: For vaults containing business-critical secrets, configure escrow enabling trusted administrators to recover contents if owners lose access.

Test vault operations before production use: Verify archival and retrieval workflows in test environments before storing production secrets.

Integration with Other IPA Components

Key Recovery Authority (KRA)

Vaults are built on Dogtag’s KRA, which provides encryption, key management, and secure storage infrastructure. KRA must be configured on at least one IPA server for vault functionality.

User and Service Management

Vault ownership and membership reference IPA users (user-* commands) and services (service-* commands), integrating secrets management with identity infrastructure.

Group Management

Groups can be vault members, enabling team-based access control that automatically adapts as group membership changes (group-* commands).

Authentication

Vault access requires Kerberos authentication. Users and services must obtain valid tickets before accessing vaults, ensuring authentication context for all vault operations.

EXAMPLES

List vaults:

ipa vault-find
    [--user <user>|--service <service>|--shared]

Add a standard vault:

ipa vault-add <name>
    [--user <user>|--service <service>|--shared]
    --type standard

Add a symmetric vault:

ipa vault-add <name>
    [--user <user>|--service <service>|--shared]
    --type symmetric --password-file password.txt

Add an asymmetric vault:

ipa vault-add <name>
    [--user <user>|--service <service>|--shared]
    --type asymmetric --public-key-file public.pem

Show a vault:

ipa vault-show <name>
    [--user <user>|--service <service>|--shared]

Modify vault description:

ipa vault-mod <name>
    [--user <user>|--service <service>|--shared]
    --desc <description>

Modify vault type:

ipa vault-mod <name>
    [--user <user>|--service <service>|--shared]
    --type <type>
    [old password/private key]
    [new password/public key]

Modify symmetric vault password:

ipa vault-mod <name>
    [--user <user>|--service <service>|--shared]
    --change-password
ipa vault-mod <name>
    [--user <user>|--service <service>|--shared]
    --old-password <old password>
    --new-password <new password>
ipa vault-mod <name>
    [--user <user>|--service <service>|--shared]
    --old-password-file <old password file>
    --new-password-file <new password file>

Modify asymmetric vault keys:

ipa vault-mod <name>
    [--user <user>|--service <service>|--shared]
    --private-key-file <old private key file>
    --public-key-file <new public key file>

Delete a vault:

ipa vault-del <name>
    [--user <user>|--service <service>|--shared]

Display vault configuration:

ipa vaultconfig-show

Archive data into standard vault:

ipa vault-archive <name>
    [--user <user>|--service <service>|--shared]
    --in <input file>

Archive data into symmetric vault:

ipa vault-archive <name>
    [--user <user>|--service <service>|--shared]
    --in <input file>
    --password-file password.txt

Archive data into asymmetric vault:

ipa vault-archive <name>
    [--user <user>|--service <service>|--shared]
    --in <input file>

Retrieve data from standard vault:

ipa vault-retrieve <name>
    [--user <user>|--service <service>|--shared]
    --out <output file>

Retrieve data from symmetric vault:

ipa vault-retrieve <name>
    [--user <user>|--service <service>|--shared]
    --out <output file>
    --password-file password.txt

Retrieve data from asymmetric vault:

ipa vault-retrieve <name>
    [--user <user>|--service <service>|--shared]
    --out <output file> --private-key-file private.pem

Add vault owners:

ipa vault-add-owner <name>
    [--user <user>|--service <service>|--shared]
    [--users <users>]  [--groups <groups>] [--services <services>]

Delete vault owners:

ipa vault-remove-owner <name>
    [--user <user>|--service <service>|--shared]
    [--users <users>] [--groups <groups>] [--services <services>]

Add vault members:

ipa vault-add-member <name>
    [--user <user>|--service <service>|--shared]
    [--users <users>] [--groups <groups>] [--services <services>]

Delete vault members:

ipa vault-remove-member <name>
    [--user <user>|--service <service>|--shared]
    [--users <users>] [--groups <groups>] [--services <services>]

Use Cases

1. Storing Database Password in User Vault

Store sensitive database password in personal symmetric vault for secure access.

# Create symmetric vault for database password
ipa vault-add db_prod_password --type symmetric

# Will prompt for vault password
# Enter strong vault password (different from IPA password)

# Store database password in file temporarily
echo -n 'MySuperSecretDBPass123!' > /tmp/db_password.txt
chmod 600 /tmp/db_password.txt

# Archive password into vault
ipa vault-archive db_prod_password \
  --in /tmp/db_password.txt

# Will prompt for vault password entered during vault creation

# Securely delete temp file
shred -u /tmp/db_password.txt

# Later, retrieve password when needed
ipa vault-retrieve db_prod_password \
  --out /tmp/db_password_retrieved.txt

# Use password and securely delete
cat /tmp/db_password_retrieved.txt  # Use in script/command
shred -u /tmp/db_password_retrieved.txt

2. Shared Team Vault for Service Account Credentials

Create shared vault accessible by entire team for shared service account password.

# Create shared vault container (if not exists)
ipa vaultcontainer-show --shared 2>/dev/null || \
  ipa vaultcontainer-add-owner --shared --users=admin

# Create shared symmetric vault
ipa vault-add team_ldap_service_account --shared --type symmetric
# Enter vault password that will be shared with team

# Archive service account password
echo -n 'ServiceAcctPass456' > /tmp/svc_password.txt
ipa vault-archive team_ldap_service_account --shared \
  --in /tmp/svc_password.txt
shred -u /tmp/svc_password.txt

# Add team members to vault
ipa vault-add-member team_ldap_service_account --shared \
  --groups=sysadmin-team

# All sysadmin-team members can now retrieve password
# Each team member retrieves with shared vault password
ipa vault-retrieve team_ldap_service_account --shared \
  --out /tmp/retrieved.txt

3. Asymmetric Vault for Maximum Security Secrets

Use asymmetric encryption for secrets requiring protection even from IPA administrators.

# Generate RSA key pair for vault
openssl genrsa -out /tmp/vault_private.pem 4096
openssl rsa -in /tmp/vault_private.pem -pubout -out /tmp/vault_public.pem

# Create asymmetric vault with public key
ipa vault-add critical_api_key --type asymmetric \
  --public-key-file /tmp/vault_public.pem

# Archive secret (encrypted with public key)
echo -n 'CriticalAPIKey789XYZ' > /tmp/api_key.txt
ipa vault-archive critical_api_key --in /tmp/api_key.txt
shred -u /tmp/api_key.txt

# Retrieve secret (requires private key)
ipa vault-retrieve critical_api_key \
  --out /tmp/api_key_retrieved.txt \
  --private-key-file /tmp/vault_private.pem

# Move private key to secure offline storage
# Store private key in physical safe or HSM
# Without private key, not even IPA admins can decrypt

cat /tmp/api_key_retrieved.txt  # Use the secret
shred -u /tmp/api_key_retrieved.txt

4. Service Vault for Application Configuration Secrets

Store application secrets in service vault for automated retrieval.

# Create service principal for application
ipa service-add myapp/app01.example.com

# Retrieve service keytab to application host
ssh app01.example.com
sudo ipa-getkeytab -s ipa01.example.com \
  -p myapp/app01.example.com \
  -k /etc/myapp/myapp.keytab

# As admin, create standard service vault
ipa vault-add config_secret --service myapp/app01.example.com \
  --type standard

# Archive application configuration secret
echo -n 'AppConfigSecret123' > /tmp/config.txt
ipa vault-archive config_secret --service myapp/app01.example.com \
  --in /tmp/config.txt
shred -u /tmp/config.txt

# On application host, service retrieves its secret
ssh app01.example.com
sudo kinit -kt /etc/myapp/myapp.keytab myapp/app01.example.com
ipa vault-retrieve config_secret --service myapp/app01.example.com \
  --out /etc/myapp/config_secret.txt
sudo chmod 600 /etc/myapp/config_secret.txt
sudo chown myapp:myapp /etc/myapp/config_secret.txt

# Application reads secret from file securely

5. Changing Vault Encryption Type

Convert vault from standard to symmetric encryption for enhanced security.

# Show current vault type
ipa vault-show my_secret

# Current Type: standard
# To convert to symmetric, modify vault type
ipa vault-mod my_secret --type symmetric \
  --new-password-file /tmp/new_vault_password.txt

# Vault contents automatically re-encrypted with new type
# Future archive/retrieve operations require vault password

# Verify change
ipa vault-show my_secret
# Type: symmetric

# Test retrieval with new password
ipa vault-retrieve my_secret --out /tmp/test.txt \
  --password-file /tmp/new_vault_password.txt

6. Delegating Vault Ownership to Team

Add team members as vault owners for shared administration.

# Create vault initially owned by creator
ipa vault-add team_certificates --shared --type standard

# Add additional owners from security team
ipa vault-add-owner team_certificates --shared \
  --users=alice,bob \
  --groups=security-team

# All owners can now manage vault:
# - Add/remove members
# - Add/remove other owners
# - Archive/retrieve secrets
# - Modify vault configuration
# - Delete vault

# Verify ownership
ipa vault-show team_certificates --shared
# Owners will show all added users/groups

7. Auditing Vault Access and Configuration

Review vault configuration and access control for security audit.

# List all user vaults for current user
ipa vault-find

# List all shared vaults
ipa vault-find --shared

# Show specific vault details
ipa vault-show sensitive_data --all

# Check vault membership
ipa vault-show sensitive_data | grep -A10 "Members:"
ipa vault-show sensitive_data | grep -A10 "Owners:"

# List vaults for specific user
ipa vault-find --user alice

# List vaults for specific service
ipa vault-find --service HTTP/webapp.example.com

# Generate vault audit report
echo "=== Vault Audit Report ===" > vault-audit.txt
echo "User Vaults:" >> vault-audit.txt
ipa vault-find --user $(ipa user-show --raw | grep uid: | awk '{print $2}') \
  >> vault-audit.txt
echo "Shared Vaults:" >> vault-audit.txt
ipa vault-find --shared >> vault-audit.txt

8. Vault Password Rotation

Change symmetric vault password periodically for security.

# Prepare new vault password
echo -n 'NewVaultPassword789' > /tmp/new_vaultpass.txt
chmod 600 /tmp/new_vaultpass.txt

# Change vault password (requires old password)
ipa vault-mod my_vault --change-password \
  --old-password-file /tmp/old_vaultpass.txt \
  --new-password-file /tmp/new_vaultpass.txt

# OR use interactive mode
ipa vault-mod my_vault --change-password
# Prompts for old password, then new password

# Vault contents remain unchanged, only password rotated
# All future archive/retrieve require new password

# Securely communicate new password to vault members
# Consider using separate vault to share vault passwords

shred -u /tmp/new_vaultpass.txt /tmp/old_vaultpass.txt

9. Rotating Asymmetric Vault Keys

Replace compromised or aged asymmetric vault key pair.

# Generate new RSA key pair
openssl genrsa -out /tmp/new_private.pem 4096
openssl rsa -in /tmp/new_private.pem -pubout -out /tmp/new_public.pem

# Re-key vault (requires old private key to decrypt, re-encrypts with new public key)
ipa vault-mod secure_vault --type asymmetric \
  --private-key-file /tmp/old_private.pem \
  --public-key-file /tmp/new_public.pem

# Vault contents re-encrypted with new public key
# Old private key no longer works
# Only new private key can retrieve secrets

# Securely archive old private key if needed for audit trail
# Then destroy: shred -u /tmp/old_private.pem

# Store new private key securely (offline, HSM, physical safe)
chmod 400 /tmp/new_private.pem
# Move to secure storage location

10. Recovering Deleted Vault Contents

Understand vault archival overwrites previous secrets.

# Vaults store only ONE secret at a time
# Archiving new secret OVERWRITES old secret

# First archival
echo "Version 1 of secret" > /tmp/secret_v1.txt
ipa vault-archive my_vault --in /tmp/secret_v1.txt

# Second archival REPLACES version 1
echo "Version 2 of secret" > /tmp/secret_v2.txt
ipa vault-archive my_vault --in /tmp/secret_v2.txt

# Retrieve returns version 2 only; version 1 is GONE
ipa vault-retrieve my_vault --out /tmp/retrieved.txt
cat /tmp/retrieved.txt
# Output: "Version 2 of secret"

# Version 1 cannot be recovered - vaults have no history

# Best practice: Use separate vaults for different secrets
# For versioned secrets, encode version in vault name:
ipa vault-add api_key_v1 --type symmetric
ipa vault-add api_key_v2 --type symmetric
# Each vault preserves its secret independently

Security Considerations

1. KRA Compromise Impact on Standard Vaults

Standard vaults encrypted with KRA master key; KRA compromise exposes contents.

KRA holds encryption keys for all standard vault secrets
Physical or remote compromise of KRA server enables secret decryption
IPA administrators with Directory Manager access can theoretically access KRA keys
Use symmetric or asymmetric vaults for secrets requiring protection from administrators
Standard vaults suitable for secrets requiring only access control, not admin-resistance

2. Lost Symmetric Vault Passwords

Forgotten symmetric vault passwords result in permanent secret loss.

No password recovery mechanism exists for symmetric vaults
Lost password renders vault contents permanently irretrievable
Document vault passwords in secure offline storage or separate escrow vault
Consider asymmetric vaults with key escrow for critical secrets
Test password recovery procedures before storing production secrets

3. Asymmetric Vault Private Key Protection

Private key security determines asymmetric vault security; compromise exposes secrets.

Private key never transmitted to IPA; remains on client systems
Key file must be protected with filesystem permissions (chmod 400)
Key compromise enables attacker to retrieve all vault secrets
Store private keys in HSM, encrypted filesystem, or physical safe
Establish key backup procedures; lost key means permanent secret loss
Regularly audit private key locations and access

4. Vault Password Distribution in Shared Vaults

Shared symmetric vaults require password distribution to all members.

Vault password must be communicated to all authorized members
Insecure password distribution (email, chat) undermines vault security
Use out-of-band secure channels for password sharing
Consider nested vaults: use asymmetric vault to store symmetric vault passwords
Password compromise requires rotating vault password for all members

5. Single Secret Limitation and Secret Rotation

Vaults store only one secret; archiving overwrites previous secret.

No version history or multiple secret storage within single vault
Old secrets permanently lost when new secret archived
Forces proper secret rotation but prevents accidental secret retention
Create separate vaults for distinct secrets, not secret versions
Consider external secret versioning if history required

6. Vault Access Logging and Monitoring

Vault access should be monitored for security incidents and compliance.

IPA logs vault retrieve operations in audit logs
Archive operations also logged with operator identity
Centralize IPA audit logs to SIEM for alerting on unusual access patterns
Monitor for: off-hours access, access by unexpected users, bulk retrieval
Unexpected vault access may indicate compromised credentials

7. Service Vault Keytab Protection

Service vault security depends on service keytab confidentiality.

Service keytab enables authentication as service to access service vaults
Keytab compromise allows attacker to retrieve all service vault secrets
Protect keytabs with strict file permissions (root:root 0600)
Rotate service keytabs regularly; monitor keytab file access
Service compromise includes vault compromise; harden service security

8. Group Membership in Vault Access Control

Using groups for vault membership creates dynamic access; membership changes affect vault access.

Adding user to group automatically grants vault access if group is vault member
Removing user from group revokes vault access
Group membership changes may not be immediately obvious to vault owners
Audit group memberships regularly to ensure vault access remains appropriate
Document which groups have vault access in group descriptions

9. Vault Container Ownership Implications

Vault container owners can create vaults within container; enables privilege escalation.

Container ownership grants ability to create vaults in that container
User vault containers owned by user; grants unrestricted personal vault creation
Shared container ownership should be restricted to vault administrators
Service container ownership should be limited to service administrators
Container owner changes should be rare and carefully controlled

10. No Built-in Vault Secrets Expiration

Vaults don’t enforce secret expiration; secrets may become stale.

Archived secrets remain indefinitely until explicitly overwritten or vault deleted
No automatic expiration, rotation, or staleness detection
Establish operational procedures for regular secret rotation
Document secret last-update date in vault description field
Review vaults periodically to identify and rotate stale secrets

11. Vault Membership Does Not Imply Vault Awareness

Users may be vault members without knowing vault exists or contains secrets for them.

Group-based membership grants access without explicit notification
Users unaware of vault existence cannot retrieve secrets they’re authorized to access
Document vault inventory and communicate existence to authorized members
Naming conventions help: shared vault names should indicate purpose
Maintain vault inventory documentation outside IPA for reference

12. Transport Security for Vault Operations

Vault operations transmit secrets between client and server; require secure channels.

All vault operations use HTTPS/TLS for transport encryption
Standard vaults rely primarily on transport encryption for in-transit security
Symmetric/asymmetric vaults add client-side encryption layer
Compromised TLS (MITM) can expose standard vault contents in transit
Ensure IPA server certificates valid and trusted; avoid cert warnings
Certificate pinning in automated tools prevents MITM attacks

13. Vault Deletion is Permanent

Deleted vaults and their contents cannot be recovered without backup.

vault-del permanently removes vault and archived secrets
No “trash” or recovery mechanism for deleted vaults
Accidental deletion by vault owner results in permanent secret loss
Maintain IPA backups regularly for vault disaster recovery
Test vault backup restoration procedures in non-production environment
Restrict vault deletion to minimal owners; use members for access-only

14. Vault Enumeration by Authenticated Users

Any authenticated user can list vaults (with limited details).

vault-find shows vault names and basic metadata to all authenticated users
Vault contents protected but vault existence not secret
Vault names may leak information about secrets (e.g., “aws_prod_credentials”)
Use generic vault names if existence should not be disclosed
Limit vault enumeration through RBAC if required (advanced configuration)

15. KRA Availability as Single Point of Failure

Vault operations require KRA availability; KRA outage prevents vault access.

All vault archive and retrieve operations require KRA
KRA service disruption prevents secret retrieval even for cached IPA users
Deploy multiple KRA replicas for high availability
Monitor KRA service health; alert on KRA failures
Maintain offline copies of critical secrets for KRA disaster recovery
Document KRA recovery procedures before KRA failure occurs

Troubleshooting

1. KRA Not Installed - Vault Commands Fail

Symptom: Vault commands return error “KRA service not found” or “vault service unavailable”.

Diagnosis:

# Check if KRA installed on any IPA server
ipa server-role-find --role "KRA server"

# If no results, KRA not installed in environment
# Check specific server
ipa server-show ipa01.example.com | grep KRA

Resolution:

Install KRA on at least one IPA server:

  ssh ipa01.example.com
  sudo ipa-kra-install

Follow prompts to configure KRA
After installation, vault commands will work
For HA, install KRA on multiple servers

2. Incorrect Vault Password - Cannot Retrieve

Symptom: vault-retrieve fails with “incorrect password” for symmetric vault.

Diagnosis:

# Verify vault is symmetric type
ipa vault-show my_vault | grep Type
# Should show: Type: symmetric

# Ensure using correct password file or interactive prompt
# Common issue: whitespace or newline in password file
hexdump -C /tmp/password.txt
# Check for unexpected characters (0x0a = newline)

Resolution:

Use echo -n when creating password files to avoid trailing newline:

  echo -n 'MyPassword' > /tmp/password.txt

Try interactive password entry instead of file:

  ipa vault-retrieve my_vault --out /tmp/secret.txt
  # Enter password when prompted

If password truly lost, vault contents permanently irretrievable
Recreate vault and re-archive secret if original secret still available

3. Private Key Does Not Match Vault Public Key

Symptom: Asymmetric vault retrieval fails with “private key does not match public key” error.

Diagnosis:

# Verify vault is asymmetric type
ipa vault-show my_vault | grep Type

# Extract public key from private key and compare
openssl rsa -in /tmp/private.pem -pubout -out /tmp/extracted_public.pem

# Compare with vault's public key
# (vault public key visible in vault-show --all output)

Resolution:

Ensure using correct private key file for this vault
If vault keys were rotated, use current private key, not old one
If private key lost, vault contents permanently irretrievable
May need to recreate vault with new key pair if original secret available
Verify private key file integrity (not corrupted)

4. Cannot Create Vault - Container Does Not Exist

Symptom: vault-add fails for shared vault with “container not found” error.

Diagnosis:

# Check if shared vault container exists
ipa vaultcontainer-show --shared

# If error, shared container not created yet

Resolution:

Create shared vault container first:

  # Admin creates shared container
  ipa vaultcontainer-add-owner --shared --users=admin

Then create vaults within container:

  ipa vault-add my_shared_vault --shared --type standard

User and service containers created automatically on first vault creation

5. Vault Membership Not Granting Access

Symptom: User added as vault member but vault-retrieve fails with permission denied.

Diagnosis:

# Verify user is member of vault
ipa vault-show vault_name | grep -A5 "Members:"

# Check if user vs owner confusion
# Members can retrieve; owners can also modify
ipa vault-show vault_name | grep -A5 "Owners:"

# Verify user authenticated with valid Kerberos ticket
klist

Resolution:

Ensure user has valid Kerberos ticket: kinit username
Verify user listed in vault members (or owners - owners are also members)
If group is member, verify user is member of that group:

  ipa group-show groupname | grep members

Wait for replication if vault membership just added on different server
Check vault container permissions if issue persists

6. Vault Archive Overwrites Existing Secret

Symptom: Archived new secret but now old secret is gone; need to recover old version.

Diagnosis:

# Vaults store only ONE secret at a time
ipa vault-show vault_name
# No version history visible

# Retrieval returns most recently archived secret only
ipa vault-retrieve vault_name --out /tmp/current.txt

Resolution:

Old secret is permanently lost; vaults have no history
Cannot recover previous secret version from vault
May be recoverable from IPA backup if backup recent enough:

  # Contact IPA administrator for backup restoration
  # ipa-backup and ipa-restore tools

Prevention: Use separate vaults for different secrets
For versioned secrets, create vault per version: secret_v1, secret_v2

7. Cannot Delete Vault - Permission Denied

Symptom: vault-del fails even though user is vault member.

Diagnosis:

# Check if user is vault owner (not just member)
ipa vault-show vault_name

# Members can retrieve; only owners can delete
# Owners section must include user

Resolution:

Only vault owners can delete vaults, members cannot
Request current owner to delete vault, or
Request current owner to add you as owner:

  ipa vault-add-owner vault_name --users=your_username

Then retry delete:

  ipa vault-del vault_name

8. Service Cannot Retrieve from Service Vault

Symptom: Service vault retrieval fails with authentication error.

Diagnosis:

# On service host, verify service keytab exists
ls -l /etc/krb5.keytab
klist -kt /etc/krb5.keytab | grep servicename

# Verify service has valid Kerberos ticket
klist
# Should show service principal, not host principal

# Test service authentication
kinit -kt /etc/krb5.keytab service/hostname
klist

Resolution:

Service must authenticate with service principal, not host principal:

  kinit -kt /etc/servicename/service.keytab service/host.example.com

Verify vault is service vault with correct service:

  ipa vault-show vault_name --service service/host.example.com

Check service is member or owner of vault
Ensure keytab file readable by service user account

9. Vault Operations Very Slow

Symptom: vault-archive and vault-retrieve commands take very long time.

Diagnosis:

# Check KRA service status and load
ssh ipa01.example.com
systemctl status pki-tomcatd@pki-tomcat

# Review KRA logs for errors
tail -100 /var/log/pki/pki-tomcat/kra/debug

# Check KRA connectivity from client
curl -v https://ipa01.example.com:443/kra/rest/

Resolution:

KRA performance issues may indicate Java heap or database problems
Check KRA Tomcat memory settings in /etc/pki/pki-tomcat/tomcat.conf
Increase KRA heap if needed: -Xmx setting
Check certificate database size and integrity
Restart KRA service: systemctl restart pki-tomcatd@pki-tomcat
If persistent, check for KRA bugs in Red Hat support portal

10. Vault Description or Metadata Not Updating

Symptom: vault-mod --desc command succeeds but description doesn’t change.

Diagnosis:

# Show vault with all attributes
ipa vault-show vault_name --all

# Check if replication delayed
ipa-replica-manage list

# Verify modification on different replica
ssh ipa02.example.com
ipa vault-show vault_name

Resolution:

Vault metadata changes may take time to replicate
Wait 30-60 seconds and re-check
If multi-master deployment, check replication status:

  ipa topologysuffix-verify domain

Force replication if needed:

  ipa-replica-manage force-sync --from=ipa01.example.com

11. Cannot List Shared Vaults

Symptom: vault-find --shared returns no results but shared vaults exist.

Diagnosis:

# Verify shared vault container exists
ipa vaultcontainer-show --shared

# Check specific vault
ipa vault-show vault_name --shared

# Verify user has permission to list shared vaults
ipa vault-find --shared --all

Resolution:

Ensure using --shared flag correctly
Shared vaults live in shared container, separate from user vaults
User must have permissions to read shared container
Check RBAC permissions if custom permission model deployed
Verify vault was created with --shared flag

12. Vault Type Modification Fails

Symptom: Changing vault type from standard to symmetric fails.

Diagnosis:

# Attempt vault type change
ipa vault-mod vault_name --type symmetric --new-password

# Check current vault state
ipa vault-show vault_name --all

Resolution:

Vault type change requires vault to be empty (no secret archived)
Retrieve and save secret before type change:

  ipa vault-retrieve vault_name --out /tmp/secret_backup.txt

Delete and recreate vault with new type:

  ipa vault-del vault_name
  ipa vault-add vault_name --type symmetric
  ipa vault-archive vault_name --in /tmp/secret_backup.txt
  shred -u /tmp/secret_backup.txt

Or use vault-mod with correct old/new credentials for type conversion

13. Group Added as Member But Members Cannot Access

Symptom: Group added to vault membership but group members cannot retrieve secrets.

Diagnosis:

# Verify group is vault member
ipa vault-show vault_name | grep -A10 "Member groups"

# Verify user is group member
ipa group-show groupname | grep members

# Check if user authenticated
klist

Resolution:

Group membership may take time to propagate through SSSD cache
User may need to re-authenticate to pick up new group membership:

  kdestroy
  kinit username

Verify user actually is member of group
Wait for SSSD cache refresh or force refresh:

  sss_cache -E

Group membership inheritance not supported; user must be direct group member

14. Vault Container Ownership Confusion

Symptom: Cannot create vault in another user’s vault container.

Diagnosis:

# Attempt to create vault for different user
ipa vault-add vault_name --user alice
# May fail if you're not alice and not container owner

# Check container ownership
ipa vaultcontainer-show --user alice

Resolution:

Only user alice (or container owners) can create vaults in alice’s container
User vault containers owned by user; others cannot create vaults there
Use shared vaults for multi-user access:

  ipa vault-add vault_name --shared --type standard

Or have alice create vault and add you as member/owner

15. Testing Vault Backup and Recovery

Symptom: Want to verify vaults included in IPA backups before disaster occurs.

Diagnosis:

# Create test vault with known content
ipa vault-add backup_test --type standard
echo "Backup test secret 12345" > /tmp/test_secret.txt
ipa vault-archive backup_test --in /tmp/test_secret.txt
shred -u /tmp/test_secret.txt

# Perform IPA backup
ssh ipa01.example.com
sudo ipa-backup

# Note backup location (e.g., /var/lib/ipa/backup/...)

Resolution:

Test restoration in non-production environment or VM
Restore IPA backup to test system:

  sudo ipa-restore /var/lib/ipa/backup/ipa-full-YYYY-MM-DD-HH-MM-SS

Verify test vault and contents restorable:

  ipa vault-retrieve backup_test --out /tmp/restored_test.txt
  cat /tmp/restored_test.txt
  # Should show: "Backup test secret 12345"

Document vault backup/restore procedures for disaster recovery
Test backups regularly; backup without testing is not a backup

Commands

vault-add-member

Usage: ipa [global-options] vault-add-member NAME [options]

Add members to a vault.

Arguments

Argument	Required	Description
`NAME`	yes	Vault name

Options

Option	Description
`--service SERVICE`	Service name of the service vault
`--shared`	Shared vault
`--user USER`	Username of the user vault
`--all`	Retrieve and print all attributes from the server. Affects command output.
`--raw`	Print entries as stored on the server. Only affects output format.
`--no-members`	Suppress processing of membership attributes.
`--users USERS`	users to add
`--groups GROUPS`	groups to add
`--services SERVICES`	services to add

vault-add-owner

Usage: ipa [global-options] vault-add-owner NAME [options]

Add owners to a vault.

Arguments

Argument	Required	Description
`NAME`	yes	Vault name

Options

Option	Description
`--service SERVICE`	Service name of the service vault
`--shared`	Shared vault
`--user USER`	Username of the user vault
`--all`	Retrieve and print all attributes from the server. Affects command output.
`--raw`	Print entries as stored on the server. Only affects output format.
`--no-members`	Suppress processing of membership attributes.
`--users USERS`	users to add
`--groups GROUPS`	groups to add
`--services SERVICES`	services to add

vault-del

Usage: ipa [global-options] vault-del NAME [options]

Delete a vault.

Arguments

Argument	Required	Description
`NAME`	yes	Vault name

Options

Option	Description
`--continue`	Continuous mode: Don’t stop on errors.
`--service SERVICE`	Service name of the service vault
`--shared`	Shared vault
`--user USER`	Username of the user vault

vault-find

Usage: ipa [global-options] vault-find [CRITERIA] [options]

Search for vaults.

Arguments

Argument Required Description

CRITERIA no A string searched in all relevant object attributes

Options

Option	Description
`--name NAME`	Vault name
`--desc DESC`	Vault description
`--type TYPE`	Vault type
`--timelimit TIMELIMIT`	Time limit of search in seconds (0 is unlimited)
`--sizelimit SIZELIMIT`	Maximum number of entries returned (0 is unlimited)
`--service SERVICE`	Service name of the service vault
`--shared`	Shared vault
`--user USER`	Username of the user vault
`--services`	List all service vaults
`--users`	List all user vaults
`--all`	Retrieve and print all attributes from the server. Affects command output.
`--raw`	Print entries as stored on the server. Only affects output format.
`--pkey-only`	Results should contain primary key attribute only (“name”)

vault-remove-member

Usage: ipa [global-options] vault-remove-member NAME [options]

Remove members from a vault.

Arguments

Argument	Required	Description
`NAME`	yes	Vault name

Options

Option	Description
`--service SERVICE`	Service name of the service vault
`--shared`	Shared vault
`--user USER`	Username of the user vault
`--all`	Retrieve and print all attributes from the server. Affects command output.
`--raw`	Print entries as stored on the server. Only affects output format.
`--no-members`	Suppress processing of membership attributes.
`--users USERS`	users to remove
`--groups GROUPS`	groups to remove
`--services SERVICES`	services to remove

vault-remove-owner

Usage: ipa [global-options] vault-remove-owner NAME [options]

Remove owners from a vault.

Arguments

Argument	Required	Description
`NAME`	yes	Vault name

Options

Option	Description
`--service SERVICE`	Service name of the service vault
`--shared`	Shared vault
`--user USER`	Username of the user vault
`--all`	Retrieve and print all attributes from the server. Affects command output.
`--raw`	Print entries as stored on the server. Only affects output format.
`--no-members`	Suppress processing of membership attributes.
`--users USERS`	users to remove
`--groups GROUPS`	groups to remove
`--services SERVICES`	services to remove

vault-show

Usage: ipa [global-options] vault-show NAME [options]

Display information about a vault.

Arguments

Argument	Required	Description
`NAME`	yes	Vault name

Options

Option	Description
`--rights`	Display the access rights of this entry (requires —all). See ipa man page for details.
`--service SERVICE`	Service name of the service vault
`--shared`	Shared vault
`--user USER`	Username of the user vault
`--all`	Retrieve and print all attributes from the server. Affects command output.
`--raw`	Print entries as stored on the server. Only affects output format.
`--no-members`	Suppress processing of membership attributes.

vaultconfig-show

Usage: ipa [global-options] vaultconfig-show [options]

Show vault configuration.

Options

Option	Description
`--transport-out TRANSPORT-OUT`	Output file to store the transport certificate
`--all`	Retrieve and print all attributes from the server. Affects command output.
`--raw`	Print entries as stored on the server. Only affects output format.

vaultcontainer-add-owner

Usage: ipa [global-options] vaultcontainer-add-owner [options]

Add owners to a vault container.

Options

Option	Description
`--service SERVICE`	Service name of the service vault
`--shared`	Shared vault
`--user USER`	Username of the user vault
`--all`	Retrieve and print all attributes from the server. Affects command output.
`--raw`	Print entries as stored on the server. Only affects output format.
`--no-members`	Suppress processing of membership attributes.
`--users USERS`	users to add
`--groups GROUPS`	groups to add
`--services SERVICES`	services to add

vaultcontainer-del

Usage: ipa [global-options] vaultcontainer-del [options]

Delete a vault container.

Options

Option	Description
`--continue`	Continuous mode: Don’t stop on errors.
`--service SERVICE`	Service name of the service vault
`--shared`	Shared vault
`--user USER`	Username of the user vault

vaultcontainer-remove-owner

Usage: ipa [global-options] vaultcontainer-remove-owner [options]

Remove owners from a vault container.

Options

Option	Description
`--service SERVICE`	Service name of the service vault
`--shared`	Shared vault
`--user USER`	Username of the user vault
`--all`	Retrieve and print all attributes from the server. Affects command output.
`--raw`	Print entries as stored on the server. Only affects output format.
`--no-members`	Suppress processing of membership attributes.
`--users USERS`	users to remove
`--groups GROUPS`	groups to remove
`--services SERVICES`	services to remove

vaultcontainer-show

Usage: ipa [global-options] vaultcontainer-show [options]

Display information about a vault container.

Options

Option	Description
`--rights`	Display the access rights of this entry (requires —all). See ipa man page for details.
`--service SERVICE`	Service name of the service vault
`--shared`	Shared vault
`--user USER`	Username of the user vault
`--all`	Retrieve and print all attributes from the server. Affects command output.
`--raw`	Print entries as stored on the server. Only affects output format.
`--no-members`	Suppress processing of membership attributes.