Barebones mail signing service, remote database, caching, read only database access

Hi,

In our mail signing service implementation, we use multiple Djigzo worker nodes connecting to an Oracle database on a remote server. We manage the database from a separate host that's not used to sign e-mails, only to manage the database. It's also the only host that has digzo-web installed.
We don't use the webmail interface, the workers are not internet-facing, we don't use djigzo to encrypt e-mails (the management host is not internet-facing, either, it's only accessed internally).

The system works fine, allows really large volume of outgoing mail to be signed, but we identified two bottlenecks in the implementation and we're wondering if it's possible to eliminate them.
1. The service connects to the remote database to collect the certificate for every single signature made. This requires a significant number of logins to the database and our database admins asked us to check if we can limit this.
2. Mail signing workers connect to the database for every signature. We tried enabling cache in our testing environment (using EhCache), but djigzo.service fails to start, apparently there's no library. I didn't try adding it manually, I assume there is a reason for not shipping cache libraries with the product. Could you shed more light on it? We're not using the service to encrypt mail, we're only using it to sign.
3. Chipermail raises CRL lookup threads to our certificate providers. Internet connections are firewalled, so they time out. We don't even need the lookup functionality on workers and if we were to use it, we would run it on the management host. Can those threads be disabled?
4. Workers failed when we tried to access the database with a read-only user. Apparently Djigzo tries to write to the database and won't work if the user has no write access. Is there a way to disable writes to the database and use the database with a read only user?

To recap:
1: Can we limit the number of simultaneous logins and/or active connections to the database?
2: Can we enable caching? We don't write to the database, so if we enabled it, caching the signing keys locally would eliminate the remote database connection bottleneck.
3: Can we disable CRL lookups and all other internet connections from djigzo? Disabling them would eliminate a few threads that open a connection to the database which would also reduce the load on the db server.
4: Can we use the database from djigzo workers in read only mode? We would like to completely disable writes to the database to prevent any potential database corruption and to improve performance by eliminating any locking mechanism in the database.

Best regards, Marek

1: Can we limit the number of simultaneous logins and/or active
connections to the database?

From you mail, it looks like you are using the gateway is a very

specific setup, i.e., only signing. Is that correct? If so you can
tweak the mail flow, i.e., remove unneeded functionality, to match you
dedicated setup.

The default mail flow is defined for the general case. So it contains a
lot of check like "is DLP enabled for the sender", "is DLP enabled for
the recipient" etc. Most checks require database access.

You can remove functionality you do not need. This will make handling
email faster and remove most database calls.

The mail flow is defined in config.xml. This is more or less a state
machine which defines how email should be handled.

The default mail flow can be viewed here
https://www.ciphermail.com/documentation/gateway-administration-guide/state-diagram.html

If you have some experience with programming, changing config.xml
should not be hard.

2: Can we enable caching? We don't write to the database, so if we
enabled it, caching the signing keys locally would eliminate the
remote database connection bottleneck.

At the moment this is not supported (I mean never tested). We are
however working hard on an update of the database layer. We will check
whether caching is supported with the new database layer.

3: Can we disable CRL lookups and all other internet connections from
djigzo? Disabling them would eliminate a few threads that open a
connection to the database which would also reduce the load on the db
server.

I think you can disable starting the background thread

In the file services.xml, the CRL store maintainer is defined as
follows:

    <bean id="threadedCRLStoreUpdater"
        class="mitm.application.djigzo.service.DjigzoServices"
        factory-method="buildThreadedCRLStoreUpdater"
        init-method="start">
        <constructor-arg ref="crlStoreUpdater"/>
        <constructor-arg ref="sessionManager"/>
        <constructor-arg ref="djigzoFactoryProperties"/>
    </bean>

If you remove `init-method="start"`, the background thread should not
be started (I have not tested this though).

4: Can we use the database from djigzo workers in read only mode? We
would like to completely disable writes to the database to prevent
any potential database corruption and to improve performance by
eliminating any locking mechanism in the database.

You cannot set the database to read only. But, I think if you tweak
config.xml to only provide the functionality you need, you can
eliminate most of the database calls.

Note that changed configuration files will be overwritten when you
update to a newer version.

Kind regards,

Martijn Brinkers

···

On Tue, 2022-09-20 at 11:38 +0000, djigzo--- via Users wrote:

Hi,

In our mail signing service implementation, we use multiple Djigzo
worker nodes connecting to an Oracle database on a remote server. We
manage the database from a separate host that's not used to sign e-
mails, only to manage the database. It's also the only host that has
digzo-web installed.
We don't use the webmail interface, the workers are not internet-
facing, we don't use djigzo to encrypt e-mails (the management host
is not internet-facing, either, it's only accessed internally).

The system works fine, allows really large volume of outgoing mail to
be signed, but we identified two bottlenecks in the implementation
and we're wondering if it's possible to eliminate them.
1. The service connects to the remote database to collect the
certificate for every single signature made. This requires a
significant number of logins to the database and our database admins
asked us to check if we can limit this.
2. Mail signing workers connect to the database for every signature.
We tried enabling cache in our testing environment (using EhCache),
but djigzo.service fails to start, apparently there's no library. I
didn't try adding it manually, I assume there is a reason for not
shipping cache libraries with the product. Could you shed more light
on it? We're not using the service to encrypt mail, we're only using
it to sign.
3. Chipermail raises CRL lookup threads to our certificate providers.
Internet connections are firewalled, so they time out. We don't even
need the lookup functionality on workers and if we were to use it, we
would run it on the management host. Can those threads be disabled?
4. Workers failed when we tried to access the database with a read-
only user. Apparently Djigzo tries to write to the database and won't
work if the user has no write access. Is there a way to disable
writes to the database and use the database with a read only user?

To recap:
1: Can we limit the number of simultaneous logins and/or active
connections to the database?
2: Can we enable caching? We don't write to the database, so if we
enabled it, caching the signing keys locally would eliminate the
remote database connection bottleneck.
3: Can we disable CRL lookups and all other internet connections from
djigzo? Disabling them would eliminate a few threads that open a
connection to the database which would also reduce the load on the db
server.
4: Can we use the database from djigzo workers in read only mode? We
would like to completely disable writes to the database to prevent
any potential database corruption and to improve performance by
eliminating any locking mechanism in the database.

Best regards, Marek

--
CipherMail email encryption
Email encryption with support for S/MIME,
OpenPGP, PDF Messenger and Webmail Messenger