java - Breaking down RSA/ECB/OAEPWithSHA-256AndMGF1Padding

Question

Java has a mode called RSA/ECB/OAEPWithSHA-256AndMGF1Padding. What does that even mean?

RFC3447, Public-Key Cryptography Standards (PKCS) #1: RSA Cryptography Specifications Version 2.1, section 7.1.2 Decryption operation says Hash and MGF are both options for RSAES-OAEP-DECRYPT. MGF is it's own function, defined in Section B.2.1 MGF1 and that has it's own Hash "option" as well.

Maybe the Hash "option" in RSAES-OAEP-DECRYPT and MGF1 are supposed to be the same or maybe they're not, it is unclear to me. If they are then I guess when you have RSA/ECB/OAEPWITHSHA-256ANDMGF1PADDING that means sha256 should be used for both. But if they're not supposed to be the same then you could have sha256 used for RSAES-OAEP-DECRYPT and, for example, sha1 used for MGF1. And if that's the case then what function is sha256 supposed to be used for? And what hash algorithm is supposed to be used for the other function?

And what does ECB mean in this context? ECB is a symmetric block cipher mode. Electronic Code Book. Maybe it's supposed to mean how Java deals with plaintext's that are larger than the modulo? Like maybe splits the plaintext into chunks that are as big as the modulo and then encrypts each one with RSA and concatenates them together? I'm just guessing..

Answer 1

The default for OAEP is to use SHA-1 for MGF1 (but see the edit on the end of this answer). Note that the hash chosen doesn't have that much impact on the security of OAEP, so mostly it will be left to this default.

We can easily test this by testing it against "OAEPPadding" and OAEPParameterSpec:

// --- we need a key pair to test encryption/decryption
KeyPairGenerator kpg = KeyPairGenerator.getInstance("RSA");
kpg.initialize(1024); // speedy generation, but not secure anymore
KeyPair kp = kpg.generateKeyPair();
RSAPublicKey pubkey = (RSAPublicKey) kp.getPublic();
RSAPrivateKey privkey = (RSAPrivateKey) kp.getPrivate();

// --- encrypt given algorithm string
Cipher oaepFromAlgo = Cipher.getInstance("RSA/ECB/OAEPWITHSHA-256ANDMGF1PADDING");
oaepFromAlgo.init(Cipher.ENCRYPT_MODE, pubkey);
byte[] ct = oaepFromAlgo.doFinal("owlstead".getBytes(StandardCharsets.UTF_8));

// --- decrypt given OAEPParameterSpec
Cipher oaepFromInit = Cipher.getInstance("RSA/ECB/OAEPPadding");
OAEPParameterSpec oaepParams = new OAEPParameterSpec("SHA-256", "MGF1", new MGF1ParameterSpec("SHA-1"), PSpecified.DEFAULT);
oaepFromInit.init(Cipher.DECRYPT_MODE, privkey, oaepParams);
byte[] pt = oaepFromInit.doFinal(ct);
System.out.println(new String(pt, StandardCharsets.UTF_8));

The code will fail with a padding related exception if you substitute "SHA-256" for the MGF1 as parameter.

The reason why the extended algorithm is needed at all is compatibility with other Cipher algorithms. Code written for e.g. "RSA/ECB/PKCS1Padding" doesn't use any parameters, let alone OAEP parameters. So without the longer string OAEP cannot function as drop in replacement.

---

The mode of operation "ECB" doesn't mean anything in this context, it should have been "None" or it should have been left out completely. You can only encrypt a single block using the RSA implementation of the SunRSA provider.

If you want to encrypt more data, create a random (AES) symmetric key and encrypt that using OAEP. Then use the AES key to encrypt your specific data. This is called a hybrid cryptosystem as it uses both asymmetric and symmetric primitives to encrypt data.

---

Note that OAEP is not supported in JDK 7 (1.7) or earlier. OAEP is included in the implementation requirements for Java runtimes since Java 8:

• RSA/ECB/OAEPWithSHA-1AndMGF1Padding (1024, 2048)
• RSA/ECB/OAEPWithSHA-256AndMGF1Padding (1024, 2048)

Some protocols may require you to use SHA-256 or SHA-512 within the padding, as SHA-1 is being deprecated for most use - even if it is not directly vulnerable for this kind of purpose.

---

EDIT: this was written mostly with Java in mind. By now many other libraries seem to take a somewhat different approach and use the same hash for the (mostly empty) label and MGF1. If you have an invalid OAEP ciphertext you should first make sure that the right "default" is being used. It is impossible to wrong any library implementation for choosing their own default; in the end it is up to the protocol to define the hashes used. Unfortunately no mandatory default exists - which is especially a problem if protocol owners forget to fully specify a configuration for the algorithms.