Catching OpenSSL misuse using CodeQL
Summary: A Trail of Bits intern developed five CodeQL queries that detect misuse of the OpenSSL libcrypto API, covering too-short cipher keys, improper engine initialization, and uncleared bignums. The queries use data flow analysis to track keys passed to EVP_EncryptInit_ex, verify engine objects reach ENGINE_init and ENGINE_set_default, and are published in a GitHub repository alongside additional Go and C++ queries.
What's compelling: OpenSSL's API accepts a too-short key as valid and reads out of bounds during cipher initialization, silently leaving data protected by a weak key - and a loaded engine that skips ENGINE_init or ENGINE_set_default can produce invalid output or fall back to a different implementation.
Read the original at Trail of Bits Blog ↗