Thoughts on xtemplate

[Sean Allred]

Hello everyone,

There was recently a conversation on TeX.SX [1] between (mainly) Joseph
Wright and myself. Please see that site for the full conversation that
prompted this.

Disclaimer: I'm a huge fan of the template idea. It is a good system
and I don't want it to see unnecessary change.

Disclaimer to the disclaimer: it's the only design management paradigm
that I've come into contact with in regards to TeX. :)

I'd like to raise attention to a possible issue with xtemplate's
design. Currently, an 'object' can receive no more than nine arguments
per TeX's syntax limitations: you cannot refer to a tenth argument in a
macro definition. That is,

\DeclareObject { foo } { 10 }

will fail. In TeX terms, this makes total sense. You cannot have more
than nine mandatory arguments for any single macro---that's just the way
it is. But speaking in terms of design, there are instances where such
an object can have more than nine arguments. In reality, this is the
decision of the document designer. There should be no such limitation
on the design.

I'll refer you to the original post for Joseph's full answer, but his
suggestion is, in my opinion, a very appropriate one: going for a
completely key--value interface on the design layer (note: not the
author layer). It would certainly remove the limitation on the number
of mandatory qualities an object may have. It would also seem to be
more befitting of the verbose clarity of the design layer to do this.

I'd like to call for thoughts on the topic. Again, I strongly recommend
you read Jospeh's response to my question [1]. I recognize that the
premise of the question might be flawed---in this instance (no pun
intended), it would likely be more appropriate to create a template with
all of the 'extras'---but the concern is valid and genuine.

All the best,
Sean

[1]: http://tex.stackexchange.com/q/196285

[Allred, Sean]

I've drawn up a syntax proposal
<https://gist.github.com/vermiculus/d8ac080f3f8c7ec2bed6#file-idea-org> (as
an Org file) on GitHub:

https://gist.github.com/vermiculus/d8ac080f3f8c7ec2bed6#file-idea-org

I'm attaching the Org file itself and the (tangled) example syntax for
those of you who use Gnus or a similar client. I've managed to *completely*
screw up my environment as far as my own email goes, unfortunately


Best,
Sean

Post by Sean Allred
Hello everyone,
There was recently a conversation on TeX.SX [1] between (mainly) Joseph
Wright and myself. Please see that site for the full conversation that
prompted this.
Disclaimer: I'm a huge fan of the template idea. It is a good system
and I don't want it to see unnecessary change.
Disclaimer to the disclaimer: it's the only design management paradigm
that I've come into contact with in regards to TeX. :)
I'd like to raise attention to a possible issue with xtemplate's
design. Currently, an 'object' can receive no more than nine arguments
per TeX's syntax limitations: you cannot refer to a tenth argument in a
macro definition. That is,
\DeclareObject { foo } { 10 }
will fail. In TeX terms, this makes total sense. You cannot have more
than nine mandatory arguments for any single macro---that's just the way
it is. But speaking in terms of design, there are instances where such
an object can have more than nine arguments. In reality, this is the
decision of the document designer. There should be no such limitation
on the design.
I'll refer you to the original post for Joseph's full answer, but his
suggestion is, in my opinion, a very appropriate one: going for a
completely key--value interface on the design layer (note: not the
author layer). It would certainly remove the limitation on the number
of mandatory qualities an object may have. It would also seem to be
more befitting of the verbose clarity of the design layer to do this.
I'd like to call for thoughts on the topic. Again, I strongly recommend
you read Jospeh's response to my question [1]. I recognize that the
premise of the question might be flawed---in this instance (no pun
intended), it would likely be more appropriate to create a template with
all of the 'extras'---but the concern is valid and genuine.
All the best,
Sean
[1]: http://tex.stackexchange.com/q/196285
--
Sean Allred

[Joseph Wright]

Post by Allred, Sean
I've drawn up a syntax proposal
<https://gist.github.com/vermiculus/d8ac080f3f8c7ec2bed6#file-idea-org> (as

One thing I think to consider is the lesson of LaTeX(2e) in that a small
number of positional-based mandatory arguments work well as a user.
That's something I'd certainly expect to see in any new code too.

The issue with mixing up mandatory and optional arguments in an
object/template set up is that this then looks less clear (how many
arguments must a TeX-like document-level interface require?). Almost
certainly the number of truly *required* arguments will remain small,
and while the case that a design should not be limited by TeX is quite
true, and the same time a design interface that fundamentally fails to
translate to a TeX-based user layer is a problem too.
--
Joseph Wright

[Sean Allred]

Post by Joseph Wright
One thing I think to consider is the lesson of LaTeX(2e) in that a small
number of positional-based mandatory arguments work well as a user.
That's something I'd certainly expect to see in any new code too.

I agree wholeheartedly; as you mentioned in your post on TeX.SX,
*nobody* wants to write anything like

\UseInstance{sectioning}{latex2e}
{<full-name>}{<TOC name>}{<header name>}...

even on the design levels. This is why I am very supportive of the
key--value syntax:

\UseInstance{sectioning}{latex2e}
{
full-name = #1,
toc-name = \IfValueTF{#1}{#1}{#2},
header-name = \IfValueTF{#1}{#1}{#2},
...
}

as it is much clearer. I've seen many, many packages that use KV-based
interfaces for setup, and I think that this is similar. As long as it
remains on the design level (the 'setup' level, if you will), it should
be fine.

Post by Joseph Wright
Almost certainly the number of truly *required* arguments will
remain small, and while the case that a design should not be limited
by TeX is quite true, and the same time a design interface that
fundamentally fails to translate to a TeX-based user layer is a
problem too.

And I don't mean to advocate that these objects should be able to have
many (9+) mandatory arguments; I'm far more concerned with the
homelessness for optional ones. The support for many mandatory
arguments is a side-effect that good interface designers will know to
avoid except where it may be appropriate (I can't think of any, but in
the interest of my own ignorance). If there is a cleaner way to provide
this support for optional arguments, that's perhaps just as
good.

All the best,
Sean

I still like the idea of a purely KV 'backend' from as a language thing.

[Will Robertson]

Post by Sean Allred
And I don't mean to advocate that these objects should be able to have
many (9+) mandatory arguments; I'm far more concerned with the
homelessness for optional ones.

I think after Frank’s experiments with sectioning, it became apparent that this was a major stumbling block. When a chapter heading can have

1. title
2. toc title
3. header title
4. label
5. unnumbered flag
6. author
7. epigraph
…

it no longer makes sense to require mandatory arguments. On the other hand I do see Joseph’s point that when you have just one or two actually mandatory arguments that writing the following is a bit ugly:

\UseInstance{crossref}{latex2e}{ label = #1 }% only one mandatory argument

* * *

A model that I think has worked quite well in the LaTeX world is how BibTeX has a notion of “mandatory” and “optional” fields for bibliography references.

Perhaps we could supplement the current xtemplate interface with a keyval-type-argument (one only per interface, of course), in which the optional/mandatory nature of each keyval item within is specified by the document design (and some may well be always mandatory).

So according to the situation you could have either

\UseInstance{…}{…}{arg one}{arg two}{arg three}

or

\UseInstance{…}{…}{ arg-one=… , arg-two=…, arg-three=… }

or even a combination of the two:

\UseInstance{…}{…}{arg one}{ arg-two=…, arg-three=… }

Any thoughts?

Will

[Sean Allred]

On the other hand I do see Joseph's point that when you have just one
or two actually mandatory arguments that writing the following is a
\UseInstance{crossref}{latex2e}{ label = #1 }

That's a fair point, but I would raise that this case is duplicated in
other style languages. In CSS, we favor consistent syntax over ease of
use:

pre .tex-comment {
color: #BBB;
}

vs. the TeX-style

pre .tex-comment #BBB

simply because the extension of this pattern yields confusing
specifications:

pre .tex-comment {
color: #BBB;
font-weight: bold;
some: made-up;
properties: for-the;
purpose-of: example;
}

would necessarily become

pre .tex-comment #BBB bold made-up for-the example;

When you think about it, it's not that extreme of an example. Like
templates, style sheets are written once and used many times over.
Clear definitions in the language are preferred over better pars in code
golf. While it's awkward in the single-property case, the consistency
is worth it. In the end, it's less for a new user to learn.

I'll take a stab at my own argument and say that, in the case of CSS,
*everything* is optional, so it's a poor choice for an example. I hope
that this doesn't detract from what I'm trying to say, though :)

So according to the situation you could have either
\UseInstance{...}{...}{arg one}{arg two}{arg three}
or
\UseInstance{...}{...}{ arg-one=... , arg-two=..., arg-three=... }
\UseInstance{...}{...}{arg one}{ arg-two=..., arg-three=... }

Were you able to take a peek at the proposal document I sent out?

https://gist.github.com/vermiculus/d8ac080f3f8c7ec2bed6#file-idea-org

Given the limitations of purely positional arguments, I feel that I make
a pretty good case for keyval-based arguments. Presuming the arguments
against positional arguments are persuasive, the language impurity
introduced by the hybrid option doesn't sit well with me. If I were a
new user to the system, I would be confused that I could name some
arguments but I wasn't able to name the positional ones. It just
feels... wrong :).

To clip a bit from the above-mentioned proposal, something like this
seems most inline with the existing paradigm:

\DeclareObjectType { name }
{
first : tokenlist ,
middle : tokenlist ,
last : tokenlist ,
first : .required ,
last : .required ,
}

or the more concise

\DeclareObjectType { name }
{
first : tokenlist! , % bang => required
middle : tokenlist , % no bang => optional
last : tokenlist! ,
}

A case that I just thought of is the case of dependent arguments ---
those arguments that are optional, but require an additional piece of
(normally optional) data to be used, such as `epithet` and
`epither-credit`, for example. Perhaps something like

\DeclareObjectType { chapter }
{
title : tokenlist! ,
epigraph : tokenlist ,
epigraph-credit : tokenlist!(epigraph) ,
}

or

\DeclareObjectType { chapter }
{
title : tokenlist ,
title : .required ,
epigraph : tokenlist ,
epigraph-credit : tokenlist ,
epigraph-credit : .requires epigraph ,
}

All the best,
Sean

[Will Robertson]

Hi Sean,

Thanks for your comments here.
Like Frank said earlier, please don’t take any lack of comment from me as disinterest — it’s unfortunately a busy time for me.

Post by Sean Allred
Were you able to take a peek at the proposal document I sent out?
https://gist.github.com/vermiculus/d8ac080f3f8c7ec2bed6#file-idea-org
Given the limitations of purely positional arguments, I feel that I make
a pretty good case for keyval-based arguments. Presuming the arguments
against positional arguments are persuasive, the language impurity
introduced by the hybrid option doesn't sit well with me.

You may well be right on this.

Post by Sean Allred
To clip a bit from the above-mentioned proposal, something like this
\DeclareObjectType { name }
{
first : tokenlist ,
middle : tokenlist ,
last : tokenlist ,
first : .required ,
last : .required ,
}

Correct me if I’m being daft, but doesn’t “requiredness” need to be checked at the template not object level? I guess I’m thinking along the lines of BibTeX here — if an unknown key is given, just ignore it.

Consider perhaps a “frontmatter" object that includes information about affiliations, biographies, photographs, and so on. Depending what type of template you choose to typeset the object, you may or may not end up using the various pieces of information.

Or perhaps both objects and templates need to have a concept of which parameters are required, and only work together if they match appropriately.

Will

[Sean Allred]

Hi Will,

Like Frank said earlier, please don't take any lack of comment from me
as disinterest --- it's unfortunately a busy time for me.

No worries at all --- I'm just happy that I've had a lull in my own work
to think about this some more. So it goes :)

Correct me if I'm being daft, but doesn't "requiredness" need to be
checked at the template not object level? [...] Depending what type of
template you choose to typeset the object, you may or may not end up
using the various pieces of information.

Oh, you're absolutely right --- I never thought about it that way. If
this idea is used, templates would definitely need to have some way of
declaring which object properties are required. But perhaps...

Or perhaps both objects and templates need to have a concept of which
parameters are required, and only work together if they match
appropriately.

...there could be a combination of the two, with the current concept of
'absolutely required' elements --- those properties without which the
object is meaningless (like a section title). I think the 'matching'
idea might be a maintenance nightmare, but I think the idea would be
useful as some sort of 'inheritance' idea.

Then again, this is implementing a feature that is emphasizing an
already existing idea (assuming templates are given a sense of required
properties). While I really like the "don't repeat yourself" of having
required properties at the object level, I'm not sure it's worth the
added language complexity. I'm not saying it's a bad idea, I'm just
saying I can't quite envision how it would play out in the end.

* * *

Perhaps something like

\DeclareObject { name }
{
first : tokenlist ,
middle : tokenlist ,
last : tokenlist ,
}

\DeclareTemplateInterface { name } { full-name }
{ first, last }
{
reversed : boolean ,
first-name-format : tokenlist ,
middle-name-format : tokenlist ,
last-name-format : tokenlist ,
}

I would note that this does fit well with the existing xtemplate
interface. Normally, we would issue

\DeclareTemplateInterface { name } { full-name } { 2 } { ... }

to indicate that there are two required arguments. The required
arguments listed in the above syntax is (conveniently) in the same
position.

My immediate concern with this is the idea of (co-)dependencies.
Perhaps something like this can express those:

\DeclareTemplateInterface { name } { full }
{ last }
{ middle : first }
{
reversed : boolean ,
first-name-format : tokenlist ,
middle-name-format : tokenlist ,
last-name-format : tokenlist ,
}

Where 'last' is the only *required* property and to provide 'middle',
you must provide 'first'. Perhaps lt3graph can be leveraged?

Just thinking out loud here. Perhaps this kind of co-dependency isn't
really needed to begin with, but I suppose it's something to keep in
mind. If this was used, there would have to be some sort of
dependency-cycle checking going on at declaration time, but that's just
a detail.

* * *

Or perhaps both objects and templates need to have a concept of which
parameters are required, and only work together if they match
appropriately.

What kind of syntax do you have in mind? It might make more sense.

* * *

Thanks again (everyone --- past, present, future) for giving this a
hearing and offering your own ideas. :)

Best,
Sean

[Mittelbach, Frank]

Post by Joseph Wright

Post by Allred, Sean
I've drawn up a syntax proposal
<https://gist.github.com/vermiculus/d8ac080f3f8c7ec2bed6#file-idea-org> (as

One thing I think to consider is the lesson of LaTeX(2e) in that a small
number of positional-based mandatory arguments work well as a user.
That's something I'd certainly expect to see in any new code too.

I agree but this is a document interface question and quite independent
of the designer layer. There is no reason, whatsoever, that the
transition from gui layer to designer layer couldn't turn positional
arguments (and or a mixture of 2e type optionals) into anything on the
designer layer, be it mandatory positional arguments or key/val pairs

Post by Joseph Wright
The issue with mixing up mandatory and optional arguments in an
object/template set up is that this then looks less clear (how many
arguments must a TeX-like document-level interface require?).

be careful with that: even the mandatory template args (that are now
positional ones) do not translate to mandatory args on the user
interface, there they might as well be optional and defaults are
generated when passing them to the template so there isn't really much
in terms or relationship of "mandatory" between the two layers.

Post by Joseph Wright
Almost
certainly the number of truly *required* arguments will remain small,
and while the case that a design should not be limited by TeX is quite
true, and the same time a design interface that fundamentally fails to
translate to a TeX-based user layer is a problem too.

The number of required args will indeed be small which is why I think
the designer layer could implement them as mandatory positional args
despite the inherited limitation of 9. However, given that this would be
a one-time parsing effort between two layers I lean towards key/val here
too as it will make the designer layer easier to read.

Neither way I see that this fundamentally fails to translate a TeX-based
user layer.

frank

[Frank Mittelbach]

Hi,

I'm very much interested in this discussion, however, right now I'm fighting headwinds and rain on the bicycle with little connectivity ... Just wanted to say that silence on my part doesn't mean "no interest"

Frank

... written on the iPad

[Mittelbach, Frank]

sorry Sean for getting back so late into this (which is actually a topic
very dear to me). I decided to comment to several of the posts in
chronological order even though it would probably more appropriate to
do a summary reply ... but ...

Post by Sean Allred
Hello everyone,
Disclaimer: I'm a huge fan of the template idea. It is a good system
and I don't want it to see unnecessary change.

we I think it was a good prototype with a number of good ideas but it is
essentially flawed and I think you put the finger into one of its wounds

Post by Sean Allred
Disclaimer to the disclaimer: it's the only design management paradigm
that I've come into contact with in regards to TeX. :)

I beg to disagree, I think there is a second one which is the ldb (see
my talk from India and there is some crude implementation of it too)

both template and ldb should get married and at least in parts remodeled
to provide a coherent concept.

Post by Sean Allred
I'd like to raise attention to a possible issue with xtemplate's
design. Currently, an 'object' can receive no more than nine arguments
per TeX's syntax limitations: you cannot refer to a tenth argument in a
macro definition. That is,
\DeclareObject { foo } { 10 }
will fail. In TeX terms, this makes total sense. You cannot have more
than nine mandatory arguments for any single macro---that's just the way
it is. But speaking in terms of design, there are instances where such
an object can have more than nine arguments. In reality, this is the
decision of the document designer. There should be no such limitation
on the design.

There is a certain history to that interface. Initially we thought that
the template concept should not just be deployed at the boundary between
presentation layer and designer layer but also internally (where these
days I believe every thing should be handled by expl3 paradigms where we
only have positional arguments). Given that and the time when that
xtemplate concept was designed speed was a very important factor (still
is to some extent) and if you do key/value repeatedly rather than only
in the transition from one layer to the next that would have slowed
things down enormously --- thus the mandatory arguments which are so
much faster to scan.

Post by Sean Allred
I'll refer you to the original post for Joseph's full answer, but his
suggestion is, in my opinion, a very appropriate one: going for a
completely key--value interface on the design layer (note: not the
author layer). It would certainly remove the limitation on the number
of mandatory qualities an object may have. It would also seem to be
more befitting of the verbose clarity of the design layer to do this.

that is probably true and I do lean towards that direction too these
days (especially as I'm now looking at template concept as something
that forms the designer layer only and thus its parsing happens once per
document element only.

However, regards of mandatory (positional) or named arguments there is
an area that I feel is even more important and that hasn't been
resolved: what exactly are the arguments that make up the signature of
an object declared by \DeclareObjectType?

The background idea of \DeclareObjectType was that I wanted to have a
specification for document elements and the "document data" that they
receive that is implemented by different templates so that you could
separate the document class specification.

Reason for having that was to enable a document written for a certain
document class to receive different formatting simply by replacing the
layout spec for the class with a new one.

In LaTeX2e we don't have any such separation: article.cls does define a
document object spec implicitly but it is mixed in by defining its
formatting. As a result if you go to a different class you can't be sure
that it codes against that same spec (and in fact often enough it
doesn't, eg amsart is close but not the same - and I'm not speaking of
the fact that it internally loads amsmath for you)

So the idea was that \DeclareObjectType would formally say how many
arguments each template for this object type would implement (checked)
but also informally would state what kind of document data these
arguments should receive (not checked). The templates would not need to
actually do something with all the data received but they should be able
to absorb them.

The problem that then showed up (and isn't resolved) is

- what are those "required" arguments?
- how do you handle variation without introducing multiple
incompatible document classes

Example:

Giving a heading (section) object type. It is fairly clear that this
"requires" a title but beyond that it gets hazy. One could argue that
something like (an alternative) TOC title or running heading title is
advisable and could be made required (after all the interface from
document level to designer layer could duplicate arguments like it does
with current 2e and one could also argue that something like number
suppression as implemented by 2e is also generally a good idea but
beyond that ... is there anything else that should be forced down into
the interface?

probably not, but then anything in addition, say a "chapter motto" would
either mean that you end up with templates implementing different
document types and are incompatible with each other or you would need to
support something like defined optional data that will be parsed by the
interfaces and then used or not used by the templates.

So my current thoughts are in the following direction:

\DeclareObjectType

(1) should declare a set of data arguments that are required to be
implemented by every template of that type

(2) it should also declare a set of data argument are optional,
i.e., a template could choose to not use them without violating the
statement that this template implements the particular data type

(3) finally for special situations there could be other data
arguments (also optional) that a template might use that are not
declared at all.

==============

Now (1) could be kept positional as I really doubt that you find any
type that requires more than 9 of those despite the fact that is an
artifical restriction. However for sake of clarity on reading a designer
spec key/values are probably the better choice.

(2) definitely has to be of type key/value

(3) also need to be key/value and offering this is really there to
account for the fact that there might be special situations or things
that haven't been thought of initially. If it turns out that several
templates implement the same key meaning of type (3) then over time one
could promote that from (3) to (2) in the object type declaration, just
to formalize the meaning -- that would be upward compatible.

Offering an exit of (3) also means that one would need to parse any set
of key/values store them somewhere and offer that to the template for
use and they could decide to ignore any or all of them.

The downside is that you can't restrict yourself against a definite list
of required and optional key names and thus for the keys of type (2) one
can't identify typos in the name, say.

If (1) is key value then then parsing of arguments could compare the
list of received values against the list of required values for the
template type and at that point already complain if something is missing.

so much for my thoughts on these matters.

frank

ps some links on ideas around the architecture and ldb etc:

http://tex.stackexchange.com/questions/45838/what-can-i-do-to-help-the-latex3-project/46427#46427

http://latex-project.org/papers/LaTeX3-architecture-2011-slides.pdf

(and/or the video of the talk - link also on the website)