Enter PEM pass phrase does not work correctly

> ruby -ropenssl -e 'OpenSSL::PKey.read(File.read(File.expand_path("~/.gem/release.pem")), nil)'
Enter PEM pass phrase: (incorrect password)
Enter PEM pass phrase: (correct password)
Enter PEM pass phrase: (correct password)
-e:1:in 'OpenSSL::PKey.read': Could not parse PKey (OpenSSL::PKey::PKeyError)
	from -e:1:in '<main>'

I'm not sure why yet.

Related to ruby/rubygems#8896.

Here is a minimal reproduction:

require "openssl"
require "tempfile"

passphrase = "secret123"

# 1. Generate RSA key with passphrase
key = OpenSSL::PKey::RSA.new(2048)
encrypted_pem = key.export(OpenSSL::Cipher.new("AES-256-CBC"), passphrase)

# Save to a temporary file
key_file = Tempfile.new("private.pem")
key_file.write(encrypted_pem)
key_file.close

puts "Private key saved to: #{key_file.path}"
puts "Passphrase: #{passphrase}"
puts
puts "Now trying to read with nil passphrase (OpenSSL will prompt):"

# 2. Try reading the key (will trigger 3 OpenSSL prompts)
OpenSSL::PKey.read(File.read(key_file.path), nil)

I reproduced this with OpenSSL 3.5.1. This is probably an issue with OpenSSL 3.0 or later.

It should ask for the passphrase only once, and if the passphrase turns out to be incorrect, it should fail immediately, similar to how it used to work with OpenSSL 1.1.1 or earlier.

It currently asks for a it 3 times because of this:

openssl/ext/openssl/ossl_pkey.c

Lines 163 to 178 in dfbbac6

    
           int selections[] = { 
        
               EVP_PKEY_KEYPAIR, 
        
               EVP_PKEY_KEY_PARAMETERS, 
        
               EVP_PKEY_PUBLIC_KEY 
        
           }; 
        
           int selection_num = (int)(sizeof(selections) / sizeof(int)); 
        
           int i, j; 
        
           for (i = 0; i < input_type_num; i++) { 
        
               for (j = 0; j < selection_num; j++) { 
        
                   pkey = ossl_pkey_read(bio, input_types[i], selections[j], pass); 
        
                   if (pkey) { 
        
                       goto out; 
        
                   } 
        
               } 
        
           }

The second and third prompt are shown when it's retrying to decode it as a public key or parameters, after it was not successfully decoded as a private key. I'm not sure why OpenSSL is attempting to decrypt the PEM block in those retries, only to discard the result, but I overlooked this behavior.

There is another issue. The passphrase is not cached in OSSL_DECODER_CTX, and if the input contains two encrypted PEM blocks, it will prompt for the passphrase for every encountered PEM block:

$ ruby -Ilib -ropenssl -e'pk=OpenSSL::PKey.generate_key("ed25519"); pem=pk.private_to_pem("aes-256-cbc", "secret123"); OpenSSL::PKey.read(pem+pem) {p :asked;"bad"}'

This calls the block 4 times.

Should we file upstream bug report to OpenSSL?

pkey: stop retrying after non-retryable error from OSSL_DECODER

#931 should fix this.

pkey: stop retrying after non-retryable error from OSSL_DECODER

[ruby/openssl] pkey: stop retrying after non-retryable error from OSS…

Thanks 🙏

pkey: stop retrying after non-retryable error from OSSL_DECODER

Navigation Menu

Repository navigation

Enter PEM pass phrase does not work correctly #927

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	int selections[] = {
	EVP_PKEY_KEYPAIR,
	EVP_PKEY_KEY_PARAMETERS,
	EVP_PKEY_PUBLIC_KEY
	};
	int selection_num = (int)(sizeof(selections) / sizeof(int));
	int i, j;

	for (i = 0; i < input_type_num; i++) {
	for (j = 0; j < selection_num; j++) {
	pkey = ossl_pkey_read(bio, input_types[i], selections[j], pass);
	if (pkey) {
	goto out;
	}
	}
	}

Enter PEM pass phrase does not work correctly #927

Description

Activity

ioquatix commented on Aug 3, 2025

rhenium commented on Aug 4, 2025

rhenium commented on Aug 4, 2025

ioquatix commented on Aug 4, 2025

rhenium commented on Aug 12, 2025

ioquatix commented on Sep 12, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions