> However, Unicode is not a very good character set, and they should not force or expect you to use it.
Unicode is an excellent character set, and for 99% of cases (much more probably) it's absolutely the best choice. So one should choose Unicode (and UTF-8) in all cases unless there is an excellent reason to do otherwise. As time passes there will be fewer and fewer cases where Unicode is not sufficient, so really we are asymptotically approaching the point at which Unicode is the only good choice to make.
This is all independent of ASN.1. But it is true that ASN.1 has built-in types for Unicode and non-Unicode strings that many other protocols lack.
Have you written up anything about ASN.1X anywhere? I'd love to take a look.
> Have you written up anything about ASN.1X anywhere? I'd love to take a look.
ASN1_BCD_STRING (64): Represents a string with the following characters:
"0123456789*#+-. " (excluding the quotation marks). Each octet encodes
two characters, where the high nybble corresponds to the first character
and the low nybble corresponds to the second character.
ASN1_PC_STRING (65): Represents a string of characters in the PC
character set. Note that the control characters can also be used as
graphic characters.
ASN1_TRON_STRING (66): Represents a string of characters in the TRON
character set, encoded as TRON-8.
ASN1_KEY_VALUE_LIST (67): Represents a set of keys (with no duplicate
keys) and with a value associated with each key. The encoding is the same
as for a SET of the keys, but with the corresponding value immediately
after each key (when they are sorted, only the keys are sorted and the
values are kept with the corresponding keys).
ASN1_UTC_TIMESTAMP (68): Represents a number of UTC seconds (and
optionally fractions of seconds), excluding leap seconds, relative to
epoch.
ASN1_SI_TIMESTAMP (69): Represents a number of SI seconds (and
optionally fractions of seconds), including leap seconds, relative to
epoch.
ASN1_UTC_TIME_INTERVAL (70): Represents a time interval as a number
of UTC seconds. The number of seconds does not include leap seconds.
ASN1_SI_TIME_INTERVAL (71): Represents a time interval as a number
of SI seconds (which may include fractions).
ASN1_OUT_OF_BAND (72): This type is not for use for general-purpose
data. It represents something which is transmitted out of band (e.g. a
file descriptor) with whatever transport mechanism is being used. The
transport mechanism defines how a value of this type is supposed to be
encoded with whatever ASN.1 encoding is being used.
ASN1_MORSE_STRING (73): Represents a string of characters in the
Morse character set. The encoding is like a relative object identifier,
where 0 means an empty space, and others is like bijective base 2 with
1 for dots and 2 for dashes, with the high bit for the first dot/dash,
e.g. 4 means A and 8 means U.
ASN1_REFERENCE (74): A reference to another node within the same file.
(Not all implementations will support this feature.) The encoding is like
a Relative Object Identifier; the first number is how many times to go to
the parent node (where 0 means the reference itself), and then the rest of
the numbers specify which child node of the current node to go to where 0
means the first child, 1 means the second child, etc. It can reference a
primitive or constructed node of a BER file, but you cannot specify a
child index for a child of a primitive node, since primitive nodes cannot
have child nodes. At least one number (how many levels of parents) is
required, but any number of numbers is potentially possible.
ASN1_IDENTIFIED_DATA (75): Data which has a format and/or meaning which
is identified within the data. The encoding is always constructed and
consists of two or three items. The first item is a set of object
identifiers, object descriptors (used only for display), and/or sequences
where the first item of the sequence is a object identifier. The receiver
ignores any items in this set that it does not understand. The second
item in a ASN1_IDENTIFIED_DATA can be any single item of any type; it is
interpreted according to the object identifiers in the first set that the
receiver understands. The third item is optional, and if it is present it
is a key/value list of extensions; the keys are object identifiers and
the values are of any type according to the object identifiers. The default
value of this key/value list is an empty key/value list.
ASN1_RATIONAL (76): Stored as constructed, containing two integers, being
the numerator and the denominator. The denominator must be greater than
zero. If it is canonical form, then it must be lowest terms.
ASN1_TRANSLATION_LIST (77): A key/value list where the keys identify
languages. If the key is null then it means the default in case no language
present in this list is applicable. The types of the values depends on the
application (usually they will be some kind of character strings).
In addition, the same number for the BMP string type can also be used for a UTF-16 string type, and there is a "OBJECT IDENTIFIER RELATIVE TO" type which encodes a OID as either relative or absolute (in canonical form, it is always relative when possible) in order to save space; the schema will specify what it is relative to. ANY and ANY DEFINED BY are allowed despite being removed from the most recent versions of standard ASN.1. (The schema format for these extensions is not defined, since I am not using the ASN.1 schema format; however, someone who does use it might do so if they need it.)
There is also SDER, which is a superset of DER but a subset of BER, in case you do not want the mess of BER but do not want to require strictly canonical form either; and also SDSER which uses the same encoding for types and values than SDER but but length works differently in order to support streaming better.
As is usual, you do not have to use any or all of these types, but someone might find them useful for some uses. I have used some of them in my own stuff.
ASN1_BCD_STRING can be just IA5String with a constraint attached...
Your time types can be just an INTEGER with a constraint attached... (In Heimdal we use INTEGER constraints to pick a representation in the programming language.) E.g.,
-- 64-bit signed count of seconds where 0 is the Unix epoch
ASN1_UTC_TIMESTAMP ::= INTEGER (-18446744073709551616..18446744073709551615)
ASN1_OUT_OF_BAND can just be a NULL with an APPLICATION tag or whatever:
Out-of-Band ::= [APPLICATION 100] NULL
or maybe an ENUMERATED or BIT STRING with named bits to indicate what kind of thing is referenced out of band. You might even use this with a SEQUENCE type instead where one member identifies an out of band datum as an index, and the other identifies the kind.
ASN1_REFERENCE is... interesting. I've not needed it, but some RPC protocols support intra-payload and even circular references, so if you have a need for that (hopefully you don't), then your ASN1_REFERENCE would be useful indeed.
ASN1_RATIONAL is just a tagged sequence of numerator and denominator, with a constraint that the denominator must not be zero.
OBJECT IDENTIFIER RELATIVE TO is just a CHOICE of OBJECT IDENTIFIER and RELATIVE IDENTIFIER.
Re: SDER... yeah, so Heimdal's codec produces DER but accepts a subset of BER for interop with OpenSSL and others. If you really want streaming then you'll want a variant of OER with fixed-length lengths (which IMO OER should have had, dammit), which then looks a lot like XDR but with different alignment and more types.
> ASN1_BCD_STRING can be just IA5String with a constraint attached...
The abstract meaning matches, but the format is differently.
> ASN1_OUT_OF_BAND can just be a NULL with an APPLICATION tag or whatever
There are some uses of having a dedicated "out of band" type, such as being able to find them regardless of the schema (e.g. it might be used by a protocol that can use data with any schemas, but allows out of band data with any schema for some reason, and might want to modify the representations of out of band data when sending it to someone else).
> ASN1_IDENTIFIED_DATA... ASN.1 has EMBEDDED-PDV, open types, and the TYPE-IDENTIFIER class -- there are many ways to do this in ASN.1
EMBEDDED-PDV and those other things are different situations than I am doing, although it is similar, the use is not quite the same. ASN1_IDENTIFIED_DATA is simpler in some ways but also allows some things that EMBEDDED-PDV does not do.
Programs can also use ASN1_IDENTIFIED_DATA to identify the schema of a file that uses this type (and potentially be able to e.g. uncompress or decrypt it; this is the reason why the identifiers are allowed to be sequences and not only plain OIDs), or a part of another file.
> ASN1_REFERENCE is... interesting. I've not needed it, but some RPC protocols support intra-payload and even circular references, so if you have a need for that (hopefully you don't), then your ASN1_REFERENCE would be useful indeed.
Yes, it is what I thought too. So far I have not needed it either, but it might sometimes be useful.
> ASN1_RATIONAL is just a tagged sequence of numerator and denominator, with a constraint...
It can be defined as such in standard ASN.1, and the format is the same as that, but the abstract meaning is different. There is also a further constraint for the canonical form.
(Currently, the only place I have used this type is the tempo ratio in the .BGM lumps in Super ZZ Zero, but it would have other uses too, such as when converting data from other formats that have a rational number type.)
> OBJECT IDENTIFIER RELATIVE TO is just a CHOICE of OBJECT IDENTIFIER and RELATIVE IDENTIFIER.
It can be implemented that way in standard ASN.1 and has the same DER representation as your described type, although the abstract meaning is essentially the same as OBJECT IDENTIFIER and there is an additional constraint in the canonical form (as far as I know, this additional constraint cannot be written in standard ASN.1, but Super ZZ Zero cares about it being in canonical form (except for sound card identifiers in .BGM lumps, but this is an implementation detail for that specific part of the program)).
> I had kind of expected a subset of x.680.
Currently I am not using the schema format for ASN.1X (nor do I use the schema format of standard ASN.1); if someone else does then they might implement a variant of X.680 for use with ASN.1X. I probably would remove some stuff (and add some stuff) if I did make a variant, though.
(The use of ASN.1X is also not defined for JER, XER, OER, etc; if someone needs to, then they might do that.)
Unicode is an excellent character set, and for 99% of cases (much more probably) it's absolutely the best choice. So one should choose Unicode (and UTF-8) in all cases unless there is an excellent reason to do otherwise. As time passes there will be fewer and fewer cases where Unicode is not sufficient, so really we are asymptotically approaching the point at which Unicode is the only good choice to make.
This is all independent of ASN.1. But it is true that ASN.1 has built-in types for Unicode and non-Unicode strings that many other protocols lack.
Have you written up anything about ASN.1X anywhere? I'd love to take a look.