Using PKCS#7 OpenSSL API #251

Muzosh · 2023-09-08T14:40:38Z

Hello!

I'm having issues running timestamping server with oqsprovider.

I have this Dockerfile:

FROM bitnami/minideb:bullseye

RUN install_packages \
    ca-certificates curl libssl-dev cmake libcivetweb-dev gcc g++ make git build-essential cmake ninja-build checkinstall zlib1g-dev wget

# OPENSSL@3
WORKDIR /root/opensslv3
RUN wget https://github.com/openssl/openssl/releases/download/openssl-3.1.2/openssl-3.1.2.tar.gz \
    && tar --strip-components=1 -zxvf openssl-3.1.2.tar.gz \
    && ./config \
    && make \
    && make install \
    && ldconfig

# OQS-PROVIDER
WORKDIR /root/oqsprovider
RUN git clone --branch 0.5.1 --depth 1 https://github.com/open-quantum-safe/oqs-provider.git .\
    && ./scripts/fullbuild.sh
RUN sed -i 's/default = default_sect/default = default_sect\noqsprovider = oqsprovider_sect\n\n\[oqsprovider_sect\]\nactivate = 1/g' /usr/local/ssl/openssl.cnf && sed -i 's/# activate = 1/activate = 1/g' /usr/local/ssl/openssl.cnf
RUN cp /root/oqsprovider/_build/lib/oqsprovider.so /usr/local/lib/ossl-modules/oqsprovider.so

# UTS-SERVER
WORKDIR /root/uts-server
RUN wget https://codeload.github.com/kakwa/uts-server/tar.gz/refs/tags/0.2.1 \
    && tar --strip-components=1 -zxvf 0.2.1\
    && cmake . \
    && make

WORKDIR /opt/uts-server

RUN groupadd -r uts-server \
    && useradd -r -g uts-server uts-server \
    && mkdir /etc/uts-server \
    && chown uts-server:uts-server /etc/uts-server

VOLUME /etc/uts-server

COPY --from=builder /root/uts-server .

EXPOSE 2020

ENTRYPOINT ["./uts-server"]

I can verify that oqsprovider is installed:

root@48d18a97c59f:/opt/uts-server# openssl list -providers
Providers:
  default
    name: OpenSSL Default Provider
    version: 3.1.2
    status: active
  oqsprovider
    name: OpenSSL OQS Provider
    version: 0.5.1
    status: active

All certificates and private keys I'm feeding it, are Dilithium5. Console openssl with oqsprovider can work with them without any issues.

In the software (using OpenSSL API), I was having issues with populating config values, proper keys, etc. But now, I'm stuck at function TS_RESP_create_response, which trows multiple OPENSSL errors (the software obtains them by repeatedly calling ERR_get_error() and printing them):

LOG_DEBUG. : OpenSSL exception: 'error:10800094:PKCS7 routines::signing not supported for this key type'
LOG_DEBUG  : OpenSSL exception: 'error:17800076:time stamp routines::pkcs7 add signature error'
LOG_DEBUG  : OpenSSL exception: 'error:17800079:time stamp routines::response setup error'

I tried to fix that by running this code before-hand (with #include <openssl/provider.h>):

OSSL_PROVIDER *oqsprovider;

oqsprovider = OSSL_PROVIDER_load(NULL, "oqsprovider");
if (oqsprovider == NULL) {
    printf("Failed to load Legacy provider\n");
    exit(EXIT_FAILURE);
}

if (ERR_get_error() == 0){
    uts_logger(ct, LOG_DEBUG, "no openssl error");
}

printf("Successfully loaded provider\n");

, which runs successfully (I see both no openssl error and Successfully loaded provider in the output. But still, this does not solve the issue and I'm still getting error:10800094:PKCS7 routines::signing not supported for this key type error.

Environment (please complete the following information):

OS: docker minideb
OpenSSL version: 3.2.1
oqsprovider version: 0.5.1

Please run the following commands to obtain the version information:

For OpenSSL: openssl version: OpenSSL 3.1.2 1 Aug 2023 (Library: OpenSSL 3.1.2 1 Aug 2023)
For oqsprovider: openssl list -providers: already posted here

Here is the certificate of a public key, associated with the private key used for signing.

I won't post private key here even if it is just a testing value, but I checked it's ASN1 structure and it's just regular PrivateKeyInfo with 1.3.6.1.4.1.2.267.7.8.7 algorithm and 7456B octet string. Also, as I said, console openssl does not have issues working with this key.

Is there something I'm missing? Is this even an issue related to OQSProvider?

The text was updated successfully, but these errors were encountered:

baentsch · 2023-09-09T05:18:36Z

Is there something I'm missing? Is this even an issue related to OQSProvider?

"Probably Yes" and "Possibly No" in that order:

You are running

OpenSSL 3.1.2 1 Aug 2023 (Library: OpenSSL 3.1.2 1 Aug 2023)

This OpenSSL version cannot handle OQS signatures in TLS operations as per documentation.

That said, I cannot be entirely certain that this is the issue here without seeing the full OpenSSL stack trace leading to this error. What you could do to test this out is use OpenSSL3.2 instead and see whether the error persists.

Muzosh · 2023-09-11T12:17:16Z

Thank you for looking into this. Unfortunately, bumping the OpenSSL version to ~~3.1.2~~ 3.2.0-alpha1 did not help.

Is there any other way for me to log openssl errors in more detail?

Currently, I do it as such:

while ((err_code = ERR_get_error()))
{
    ERR_load_TS_strings();
    uts_logger(ct, LOG_ERR, "---\nERR_error_string: '%s'\nERR_reason_error_string: '%s'\nERR_lib_error_string: '%s'\n---\n",
                ERR_error_string(err_code, NULL), ERR_reason_error_string(err_code), ERR_lib_error_string(err_code));
}

I'm not really familiar with OpenSSL C API and I can't find any documentation for it.

baentsch · 2023-09-11T12:46:28Z

Unfortunately, bumping the OpenSSL version to 3.1.2 did not help.

You need to bump to 3.2.

Any 3.1 variant doesn't have all the required capabilities.

I'm not really familiar with OpenSSL C API and I can't find any documentation for it.

Start from https://www.openssl.org/docs/manmaster/man3/ERR_print_errors.html

Muzosh · 2023-09-11T12:48:51Z

Unfortunately, bumping the OpenSSL version to 3.1.2 did not help.

My mistake, I bumped it to 3.2, I just messed up my comment here on GitHub :)

baentsch · 2023-09-11T13:22:09Z

Unfortunately, bumping the OpenSSL version to 3.1.2 did not help.

My mistake, I bumped it to 3.2, I just messed up my comment here on GitHub :)

OK -- then that isn't the issue. The API you're calling TS_RESP_create_response: What is that doing? I didn't find any documentation about it. I guess what would be best is code to allow us to reproduce the problem.

baentsch · 2023-09-12T05:00:20Z

Just did a quick API check: This seems to activate the PKCS7 logic of OpenSSL. I am unsure as to whether this is fully provider enabled. At least, no reference to providers are made there -- unlike in the code for CMS, which does work OK with provider-based signature algorithms. Maybe @levitte can provide insight?

Muzosh · 2023-09-12T05:55:02Z

I am unsure as to whether this is fully provider enabled.

Is this related only to oqsprovider? I.e. would using OQS-OpenSSL@1.1 fork solve this (I'm aware it is discontinued)?

Muzosh · 2023-09-12T11:31:12Z

So I tried it with OQS-OpenSSL@1.1.1-stable fork and I get two more detailed errors:

error:1012F040:elliptic curve routines:pkey_oqs_ctrl:fatal
error:21081093:PKCS7 routines:PKCS7_SIGNER_INFO_set:signing ctrl failure
error:2F088076:time stamp routines:ts_RESP_sign:pkcs7 add signature error
error:2F07A079:time stamp routines:TS_RESP_create_response:response setup error

root@86cedb6eb6d2:~/uts-server# openssl version -a
OpenSSL 1.1.1u  30 May 2023, Open Quantum Safe 2023-07
built on: Tue Sep 12 11:11:38 2023 UTC
platform: linux-aarch64
options:  bn(64,64) rc4(char) des(int) idea(int) blowfish(ptr) 
compiler: gcc -fPIC -pthread -Ioqs/include -Wa,--noexecstack -Wall -O3 -DOPENSSL_USE_NODELETE -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_BN_ASM_MONT -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DVPAES_ASM -DECP_NISTZ256_ASM -DPOLY1305_ASM -DNDEBUG
OPENSSLDIR: "/usr/local/ssl"
ENGINESDIR: "/usr/local/lib/engines-1.1"
Seeding source: os-specific

levitte · 2023-09-13T12:23:42Z

Unfortunately, PKCS#7 and CMS aren't fully supported for use with providers... yet.
I'm currently writing a design for the missing bits. It's too late for them to appear in 3.2, unfortunartely, but I hope to be able to target 3.3.

Muzosh · 2023-09-13T12:35:59Z

@levitte Thank you for the response! I completely understand. Would you be able to provide at least non-specific timeframe for OpenSSLv3.3 or PKCS7+CMS functionality? Based on available info, I'm guessing Q1-2 of 2024 to start exploring this?

levitte · 2023-09-13T12:57:26Z

Q2-2024 is a good target to start exploring, yeah. Or if you feel adventurous, have a look at what happens on https://github.com/openssl/openssl

baentsch · 2023-09-13T13:06:12Z

Unfortunately, PKCS#7 and CMS aren't fully supported for use with providers... yet.

Thanks for the information. CMS does work reasonably well with oqsprovider, though: Which features are missing? Please let me know if there are concrete things where I could be of assistance.

levitte · 2023-09-13T13:20:34Z

The concrete things that's missing is a replacement mechanism for these ctrls: EVP_PKEY_CTRL_CMS_ENCRYPT, EVP_PKEY_CTRL_CMS_DECRYPT and EVP_PKEY_CTRL_CMS_SIGN.
Their intent is really to pass the AlgorithmIdentifier param to the backend (legacy / engine) implementation and having it deal with them in whatever way they see fit. Those ctrls do so by passing the whole CMS-something structure and letting the backend have free reins on it, which isn't at all a supported way of doing things with providers (and in my very frank opinion, is a hack that should never have seen the light of day).

So basically, that's the bit that needs a better replacement to work with providers.

levitte · 2023-09-13T13:22:13Z

There are similar ctrls for PKCS7, which I believe is what's hitting @Muzosh

baentsch · 2023-09-13T13:48:00Z

The concrete things that's missing is a replacement mechanism for these ctrls: EVP_PKEY_CTRL_CMS_ENCRYPT, EVP_PKEY_CTRL_CMS_DECRYPT and EVP_PKEY_CTRL_CMS_SIGN.

Understood (for encrypt/decrypt: I never tested that with oqsprovider); but we do test sign/verify: Why does this work if "EVP_PKEY_CTRL_CMS_SIGN" is not properly wired up for providers? Or is my test wrong?

levitte · 2023-09-13T13:52:28Z

I actually don't know. I would need some test cases that should trigger them, and time.

baentsch changed the title ~~Is this provider supposed to work with OpenSSL API?~~ Using PKCS#7 OpenSSL API Sep 12, 2023

baentsch added the upstream openssl dependency label Oct 15, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Using PKCS#7 OpenSSL API #251

Using PKCS#7 OpenSSL API #251

Muzosh commented Sep 8, 2023 •

edited

baentsch commented Sep 9, 2023

Muzosh commented Sep 11, 2023 •

edited

baentsch commented Sep 11, 2023

Muzosh commented Sep 11, 2023

baentsch commented Sep 11, 2023

baentsch commented Sep 12, 2023

Muzosh commented Sep 12, 2023

Muzosh commented Sep 12, 2023 •

edited

levitte commented Sep 13, 2023

Muzosh commented Sep 13, 2023 •

edited

levitte commented Sep 13, 2023

baentsch commented Sep 13, 2023

levitte commented Sep 13, 2023

levitte commented Sep 13, 2023

baentsch commented Sep 13, 2023

levitte commented Sep 13, 2023

Using PKCS#7 OpenSSL API #251

Using PKCS#7 OpenSSL API #251

Comments

Muzosh commented Sep 8, 2023 • edited

baentsch commented Sep 9, 2023

Muzosh commented Sep 11, 2023 • edited

baentsch commented Sep 11, 2023

Muzosh commented Sep 11, 2023

baentsch commented Sep 11, 2023

baentsch commented Sep 12, 2023

Muzosh commented Sep 12, 2023

Muzosh commented Sep 12, 2023 • edited

levitte commented Sep 13, 2023

Muzosh commented Sep 13, 2023 • edited

levitte commented Sep 13, 2023

baentsch commented Sep 13, 2023

levitte commented Sep 13, 2023

levitte commented Sep 13, 2023

baentsch commented Sep 13, 2023

levitte commented Sep 13, 2023

Muzosh commented Sep 8, 2023 •

edited

Muzosh commented Sep 11, 2023 •

edited

Muzosh commented Sep 12, 2023 •

edited

Muzosh commented Sep 13, 2023 •

edited