Cryptography
Updated Wed, 22 Jun 2022 18:54:05 GMT

# Preferred SHA-2 hash algorithm for MGF1 in RSA

Does it make any difference to the security and efficiency if we use SHA-256 or SHA-512 for the Mask Generation Function MGF1 that generates the masking / padding within the OAEP encryption scheme and the PSS signature scheme?

A lot of implementations also seem to sport support for SHA-224 and SHA-384 for MGF1. Does it make any sense to use these truncated hash functions for something that needs to generate a mask of a particular size (close to the size of the key size for RSA)? Or would that simply add unnecessary overhead?

It seems to me that SHA-512 would be more efficient because of the output size alone. However, most implementations seem to default on SHA-1, SHA-256 or the same hash as used to hash the data for PSS.

## Solution

Since the input sizes are fixed, length-extension attacks are not relevant, so any of the SHA-2 functions reasonably implements the random oracle model assumed by OAEP or PSS via MGF1even the default of SHA-1 works with MGF1.

Obviously it will cost slightly more to use SHA-224 or SHA-384 than to use SHA-256 or SHA-512 because SHA-224 and SHA-384 are effectively truncations of SHA-256 and SHA-512: to get the same amount of output as (say) twelve SHA-384 invocations each costing a SHA-512 computation, you could pay for a mere nine SHA-512 computations instead. So the truncated options don't provide any benefit. And, of course, SHA-512 than SHA-256 is generally faster on CPUs with 64-bit adders.

All that said, it is hard to imagine that this could substantially affect security or performance since you're about to do a 2048-bit modular exponentiation anyway, which will be the bulk of the costbut, of course, you're in a better position to do that measurement in your application! (Make sure to choose $$e = 3$$ to minimize time spent in exponentiation.)