SQLCipher's OpenSSL provider fetches `EVP_MAC` / `EVP_CIPHER` from the global method store on every page operation (performance / OpenSSL 3.x best practice) · Issue #597 · sqlcipher/sqlcipher · GitHub
Skip to content

SQLCipher's OpenSSL provider fetches EVP_MAC / EVP_CIPHER from the global method store on every page operation (performance / OpenSSL 3.x best practice) #597

Description

@josefguenther

Summary

SQLCipher's OpenSSL provider (src/crypto_openssl.c) fetches its cryptographic algorithms from OpenSSL's default-libctx global method store on every single page operation, rather than fetching once and caching the handles in the codec/provider context:

  • sqlcipher_openssl_hmac (src/crypto_openssl.c) calls EVP_MAC_fetch(NULL, "HMAC", NULL) and the matching EVP_MAC_free per HMAC — i.e. per page.
  • sqlcipher_openssl_cipher calls EVP_CipherInit_ex(..., EVP_aes_256_cbc(), ...) per page; the first use of that cipher resolves an implicit fetch from the same global method store.

OpenSSL 3.x's own documentation explicitly recommends fetching algorithms once and reusing the resulting EVP_MAC / EVP_CIPHER objects, precisely because each fetch is a relatively expensive, reference-counted lookup into a process-global store (the "explicit fetching" performance guidance in the OpenSSL 3.x migration notes and the EVP_MAC_fetch(3) / EVP_CIPHER_fetch(3) man pages).

Why it matters

  1. Performance. Fetching per page (instead of per codec context) is the well-known OpenSSL-3 explicit-fetch anti-pattern. A database doing thousands of page reads/writes performs thousands of redundant global-store lookups plus EVP_MAC / EVP_MAC_CTX allocations and frees that could be amortized to a single fetch per connection. On large databases this is measurable per-operation overhead.

  2. Lock contention under concurrency. Those per-page fetches also repeatedly take OpenSSL's internal method-store lock. Under heavy multi-threaded, multi-connection page I/O this is avoidable contention that a fetch-once-and-cache approach eliminates.

No correctness / thread-safety claim: in our testing the per-page fetch lookups are properly serialized by OpenSSL's own internal lock and did not race or corrupt — this is a performance + best-practice issue, not a crash report.

Affected versions / environment

  • SQLCipher: present through the current release 4.16.0 (the OpenSSL provider src/crypto_openssl.c is unchanged across 4.14.0–4.16.0); observed on 4.14.0 community (SQLCIPHER_CRYPTO_OPENSSL, via libsqlite3-sys 0.38).
  • OpenSSL: 3.x (vendored), default library context.
  • Threading: SQLITE_THREADSAFE=1; multi-threaded application with many keyed connections doing concurrent page I/O.

Suggested fix

Fetch the EVP_MAC ("HMAC") and EVP_CIPHER ("AES-256-CBC", plus any digests) once — at provider activation or codec-context initialization (sqlcipher_codec_ctx_init / the provider context) — store the fetched handles, and reuse them for every page op:

  • pass the cached EVP_MAC* to EVP_MAC_CTX_new(...) per op instead of re-EVP_MAC_fetch-ing;
  • pass the cached EVP_CIPHER* to EVP_CipherInit_ex2(...) instead of the legacy EVP_aes_256_cbc() convenience (which re-resolves through the default libctx).

Free the cached objects at context/provider teardown. This matches OpenSSL 3.x best practice, eliminates the per-page global-store lookups (a measurable throughput win on large databases), and removes the per-page lock traffic on the shared store.

Impact

Per-page CPU overhead on every encrypted read and write, plus avoidable contention on OpenSSL's global method store under multi-threaded use. Fixable entirely within SQLCipher's OpenSSL provider, with no change to the on-disk format or the public API.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions