Djigzo with James 2.3.2 (recreating of message-IDs)

Hello

today i found that james until version 2.3.1 did alter the message-ID
by default of all messages passing by. This is undesired for a MTA as
it prevents correlating messages and there answers (reply). More
information can be found here:

http://issues.apache.org/jira/browse/JAMES-875

As far as i can tell Djigzo uses james 2.3.1 which by default
re-create the ID. Is it easily possible to update james to 2.3.2
without the need for a new Djigzo release?

Many Thanks

Andreas

The main question is "should a modified message get a new message-id"?
You could argue that when you send a message to one recipient that the
message should keep the same message-id even though the message is
encrypted or signed by the gateway. My view is that the message was
altered and the message should therefore get a new message-id. Now to
make things a bit more complicated, what should the message-id be when a
message is sent to multiple recipients with different encryption
requirements. For example recipient A cannot read encrypted or signed
email so no encryption, recipient B requires S/MIME encrypted email,
recipient C only wants digitally signed email and recipient D wants to
receive it encrypted by PDF. Djigzo splits the message into 4 different
messages because the 4 recipients have non-compatible requirements. Do
you expect all the 4 messages to have the same message-id? My view would
be that every message should have a different message-id.

Different messages with the same message-id can lead to problems. For
example Exchange has a filter that removes messages with a duplicate
message-id. This means that messages can sometimes magically 'disappear'.

A possible 'solution' I have been thinking of, but I have not yet
implemented, is to add a "X-Original-Message-ID" header that contains
the message ID of the source message.

Note that only messages that are modified (signed/encrypted etc.) have a
new message-id.

Kind regards,

Martijn Brinkers

lst_hoe02(a)kwsoft.de wrote:

···

Hello

today i found that james until version 2.3.1 did alter the message-ID
by default of all messages passing by. This is undesired for a MTA as
it prevents correlating messages and there answers (reply). More
information can be found here:

[JAMES-875] Message-ID changed by mailets (MSGID_FROM_MTA_HEADER changed) - ASF JIRA

As far as i can tell Djigzo uses james 2.3.1 which by default
re-create the ID. Is it easily possible to update james to 2.3.2
without the need for a new Djigzo release?

Many Thanks

Andreas

_______________________________________________
Users mailing list
Users(a)lists.djigzo.com
http://lists.djigzo.com/lists/listinfo/users

--
Djigzo open source email encryption

Exchange duplicate detection:

Duplicate detection of delivered messages is done by the Exchange store.
The store does duplicate detection based on two properties on the
message - the Internet Message Id and the client submit time. We would
have liked to do duplicate detection based solely on the Internet
Message Id but there are several not-to-be named applications out there
that use the same Internet Message Id on all their messages.

Martijn Brinkers wrote:

···

The main question is "should a modified message get a new message-id"?
You could argue that when you send a message to one recipient that the
message should keep the same message-id even though the message is
encrypted or signed by the gateway. My view is that the message was
altered and the message should therefore get a new message-id. Now to
make things a bit more complicated, what should the message-id be when a
message is sent to multiple recipients with different encryption
requirements. For example recipient A cannot read encrypted or signed
email so no encryption, recipient B requires S/MIME encrypted email,
recipient C only wants digitally signed email and recipient D wants to
receive it encrypted by PDF. Djigzo splits the message into 4 different
messages because the 4 recipients have non-compatible requirements. Do
you expect all the 4 messages to have the same message-id? My view would
be that every message should have a different message-id.

Different messages with the same message-id can lead to problems. For
example Exchange has a filter that removes messages with a duplicate
message-id. This means that messages can sometimes magically 'disappear'.

A possible 'solution' I have been thinking of, but I have not yet
implemented, is to add a "X-Original-Message-ID" header that contains
the message ID of the source message.

Note that only messages that are modified (signed/encrypted etc.) have a
new message-id.

Kind regards,

Martijn Brinkers

lst_hoe02(a)kwsoft.de wrote:

Hello

today i found that james until version 2.3.1 did alter the message-ID
by default of all messages passing by. This is undesired for a MTA as
it prevents correlating messages and there answers (reply). More
information can be found here:

[JAMES-875] Message-ID changed by mailets (MSGID_FROM_MTA_HEADER changed) - ASF JIRA

As far as i can tell Djigzo uses james 2.3.1 which by default
re-create the ID. Is it easily possible to update james to 2.3.2
without the need for a new Djigzo release?

Many Thanks

Andreas

_______________________________________________
Users mailing list
Users(a)lists.djigzo.com
http://lists.djigzo.com/lists/listinfo/users

--
Djigzo open source email encryption

Zitat von Martijn Brinkers <martijn(a)djigzo.com>:

The main question is "should a modified message get a new message-id"?
You could argue that when you send a message to one recipient that the
message should keep the same message-id even though the message is
encrypted or signed by the gateway. My view is that the message was
altered and the message should therefore get a new message-id. Now to
make things a bit more complicated, what should the message-id be when a
message is sent to multiple recipients with different encryption
requirements. For example recipient A cannot read encrypted or signed
email so no encryption, recipient B requires S/MIME encrypted email,
recipient C only wants digitally signed email and recipient D wants to
receive it encrypted by PDF. Djigzo splits the message into 4 different
messages because the 4 recipients have non-compatible requirements. Do
you expect all the 4 messages to have the same message-id? My view would
be that every message should have a different message-id.

message-ID is a MUA thing and identifies a message by the means of the
MUA. It is used to correlate answers, so yes, the message-ID should be
the same for all instances of the same outgoing message. A MTA should
never alter the message-ID, the only option should be to create one if
its missing.

Different messages with the same message-id can lead to problems. For
example Exchange has a filter that removes messages with a duplicate
message-id. This means that messages can sometimes magically 'disappear'.

The duplicate suppression based on the message-ID should only be
applied per final recipient (mailbox) all other schemas would be badly
broken.

A possible 'solution' I have been thinking of, but I have not yet
implemented, is to add a "X-Original-Message-ID" header that contains
the message ID of the source message.

Any insight which client actually use such a header?

Note that only messages that are modified (signed/encrypted etc.) have a
new message-id.

But this is the whole intention of Djigzo, isn't it?

Fact is that the actual behavior makes djigzo unusable for us because
our sending system depends on the message-ID to get a threading like
mail presentation.

Many Thanks

Andreas

···

From my point of view and what can be read from RFC 2822 the

Zitat von Martijn Brinkers <martijn(a)djigzo.com>:

Exchange duplicate detection:

Duplicate detection of delivered messages is done by the Exchange store.
The store does duplicate detection based on two properties on the
message - the Internet Message Id and the client submit time. We would
have liked to do duplicate detection based solely on the Internet
Message Id but there are several not-to-be named applications out there
that use the same Internet Message Id on all their messages.

Microsoft Learn: Build skills that open doors in your career

This is broken by design. The message-ID is a MUA thing and therefore
should never be changed by any MTA. If Exchange does not use the
recipient mailbox *and* the message-ID to supress duplicates the
developer have probably not read any RFC. But maybe they have learned
since 2004??

From RFC 2822 section 3.4.6

In all cases, it is the meaning that the sender of the message wishes
to convey (i.e., whether this is the same message or a different
message) that determines whether or not the "Message-ID:" field
changes, not any particular syntactic difference that appears (or does
not appear) in the message.

Regards

Andreas

This is broken by design. The message-ID is a MUA thing and therefore
should never be changed by any MTA. If Exchange does not use the
recipient mailbox *and* the message-ID to supress duplicates the
developer have probably not read any RFC. But maybe they have learned
since 2004??

Whether or not Exchange does not right thing in this case is debatable
but that's how they implemented it.

From RFC 2822 section 3.4.6

In all cases, it is the meaning that the sender of the message wishes
to convey (i.e., whether this is the same message or a different
message) that determines whether or not the "Message-ID:" field
changes, not any particular syntactic difference that appears (or does
not appear) in the message.

Correct me if I'm wrong but with "sender" they do not mean the person
that created the message. With sender they mean the sender that changes
the messages (in this case the Djigzo server).

···

--
Djigzo open source email encryption

lst_hoe02(a)kwsoft.de wrote:

Fact is that the actual behavior makes djigzo unusable for us because
our sending system depends on the message-ID to get a threading like
mail presentation.

You can update to a new James. However, I have added some functions to
James to make it throttle the connections when the queue becomes too
large so if you live without these additions you can use a different
James version. I will see whether I can add a system setting that
ensures that the message will not use a new message-id. If I add this
setting it will not be on by default because it can be problematic with
Exchange (see other messages)

Kind regards,

Martijn

···

--
Djigzo open source email encryption

Zitat von Martijn Brinkers <martijn(a)djigzo.com>:

This is broken by design. The message-ID is a MUA thing and therefore
should never be changed by any MTA. If Exchange does not use the
recipient mailbox *and* the message-ID to supress duplicates the
developer have probably not read any RFC. But maybe they have learned
since 2004??

Whether or not Exchange does not right thing in this case is debatable
but that's how they implemented it.

I would rather live with exchange users loose mail (which it does
anyway by different reasons), than to broken e-mail for the rest.

But of course this could be a config setting with a short warning.

From RFC 2822 section 3.4.6

In all cases, it is the meaning that the sender of the message wishes
to convey (i.e., whether this is the same message or a different
message) that determines whether or not the "Message-ID:" field
changes, not any particular syntactic difference that appears (or does
not appear) in the message.

Correct me if I'm wrong but with "sender" they do not mean the person
that created the message. With sender they mean the sender that changes
the messages (in this case the Djigzo server).

As far as i can see the "sender" is the *creator* of the message. With
"creator" being the one who created the content. I would not say that
there is any hint that the actual encoding of the message should alter
the ID. This should apply for all non-MIME --> MIME, 8Bit --> 7Bit or
whatever transformations. The solely purpose of the message-ID is to
give the creator (human) a possibility to include a machine parseable
context-tracker IMMO.
I fail to see any other usage for a message-ID otherwise.

Regards

Andreas

Zitat von Martijn Brinkers <martijn(a)djigzo.com>:

lst_hoe02(a)kwsoft.de wrote:

Fact is that the actual behavior makes djigzo unusable for us because
our sending system depends on the message-ID to get a threading like
mail presentation.

You can update to a new James. However, I have added some functions to
James to make it throttle the connections when the queue becomes too
large so if you live without these additions you can use a different
James version. I will see whether I can add a system setting that
ensures that the message will not use a new message-id. If I add this
setting it will not be on by default because it can be problematic with
Exchange (see other messages)

From what i have read in the james docu the non-altering behaviour
would be default from james 2.3.2 onward. This, on the other hand
means that there *is* a setting to turn it on again. Maybe the easiest
way would be to use james 2.3.2 with next Djigzo release and include
the setting in the documentation. But i don't know how much the djigzo
internal james differs from the official releases.

Regards

Andreas

Zitat von lst_hoe02(a)kwsoft.de:

Zitat von Martijn Brinkers <martijn(a)djigzo.com>:

The main question is "should a modified message get a new message-id"?
You could argue that when you send a message to one recipient that the
message should keep the same message-id even though the message is
encrypted or signed by the gateway. My view is that the message was
altered and the message should therefore get a new message-id. Now to
make things a bit more complicated, what should the message-id be when a
message is sent to multiple recipients with different encryption
requirements. For example recipient A cannot read encrypted or signed
email so no encryption, recipient B requires S/MIME encrypted email,
recipient C only wants digitally signed email and recipient D wants to
receive it encrypted by PDF. Djigzo splits the message into 4 different
messages because the 4 recipients have non-compatible requirements. Do
you expect all the 4 messages to have the same message-id? My view would
be that every message should have a different message-id.

From my point of view and what can be read from RFC 2822 the
message-ID is a MUA thing and identifies a message by the means of the
MUA. It is used to correlate answers, so yes, the message-ID should be
the same for all instances of the same outgoing message. A MTA should
never alter the message-ID, the only option should be to create one if
its missing.

This may be questionable in case we convert the message to a encrypted
PDF with attachment though. In this case a direct reply which give the
sender MUA the chance to create a useful "mail-thread" view is not
possible anyway.

Regards

Andreas