Layering in DFDL

The book "ASN.1 Complete" by Larmouth (ISBN 0-12-233435-3 and available online as a pdf) discusses the importance of 
layer support in format descriptions. 

(Begin Excerpt)

The layering concept is perhaps most commonly associated with the
International Standards Organization (ISO) and International
Telecommunications Union (ITU) "architecture" or "7-layer model" for
Open Systems Interconnection (OSI) shown in Figure 3.  While many of
the protocols developed within this framework are not greatly used
today, it remains an interesting academic study for approaches to
protocol specification. In the original OSI concept in the late 1970s,
there would be just 6 layers providing (progressively richer) carrier
services, with a final "application layer" where each specification
supported a single endapplication, with no "holes".

It became apparent, however, over the next decade, that even in the
"application layer" people wanted to leave "holes" in their
specification for later extensions, or to provide a means of tailoring
their protocol to specific needs. For example, one of the more recent
and important protocols - Secure Electronic Transactions (SET) -
contains a wealth of fully-defined message semantics, but also
provides for a number of "holes" which can transfer "merchant details"
which are not specified in the SET specification itself. So we have
basic messages for purchase requests and responses, inquiry requests
and responses, authorization requests and responses, and so on, but
within those messages there are "holes" for "message
extensions" - additional information specific to a particular
merchant.

It is thus important that any mechanism or notation for specifying a
protocol should be able to cater well for the inclusion of
"holes". This has been one of the more important developments in ASN.1
in the last decade, and will be a subject of much further discussion
in this book.

"Catering well" for the inclusion of "holes" implies that the notation
must have defined mechanisms (preferably uniformly applied to all
specifications written using that notation) to identify the contents
of a hole at communications time. (In lower layers, this is sometimes
referred to as the "protocol id" problem). Equally important, however,
are notational means to clearly identify that a specification is
incomplete (contains a hole), together with well-defined mechanisms to
relate the (perhaps later in time) specification of the contents of
holes to the location of the holes themselves.

(End Excerpt)

The argument made here is equally true for DFDL. We need the ability to describe a data format containing a hole or 
payload which another DFDL schema can then describe the format of.

DFDL actually encounters some data formats where there are discontiguous holes. Consider the nonVSAM VS format. (see "
IBM OS/390 DFSMS: Using Data Sets" IBM publication  SC26-7339-01, Second Edition, December 2000. (online at: http://
publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/DGT1D411/CCONTENTS?SHELF=EZ239126&DN=SC26-7339-01&DT=20001014144419)
 In this format, data records with actual data fields of interest are broken up into segments. The segments are of 
variable size, and a record can fit in a single segment or can span multiple segments. The segments are of 3 types, 
initial, middle (there can be zero or more of these), and final. The hole that the record fits in is assembled by 
putting together the partial holes from each of the segments. Adding minor additional complexity is that in the actual 
format the segments are then grouped into variable-sized blocks as an I/O transfer-unit efficiency optimization. 

A further wrinkle on holes in DFDL is the notion of encoding. Modern data formats often contain holes (or we'll also 
call them payloads) which have been encoded to allow data transfer in text-only mediums, or to compress to save space, 
or to encrypt, or for various other reasons. The encoding must be decoded and the resulting data is...