CardWorld versions 1 and 1a, May and August 2010 – Author: James A. Mason
http://www.yorku.ca/jmason/asdindex.htm
CardWorld illustrates use of ASDParser with an ASD grammar (Cardgram1.grm) to implement an application that understands English in limited ways. It is the smallest example which I have been able to construct that incorporates syntax, semantics, and pragmatics all in non-trivial ways. Thus it allows easy exploration of how those three components of language understanding interact. Also, the pragmatic domain of playing cards is rich enough that this model of language understanding can be extended in many interesting ways, syntactically, semantically and pragmatically.
CardWorld permits a user to interact with images of ordinary playing cards on top of a card table. Cards and piles of cards on the table can be manipulated directly with a mouse or other pointing device, and they can also be manipulated by giving commands in English to a “card agent” implicit in the Card World. A pile of cards consists of cards whose images overlap. When a pile of cards has been stacked by the card agent, the images of the cards are displayed on top of each other, slightly offset, so that only the upper face of the top card in the pile is visible, but a side and end of the pile are visible too. Initially CardWorld shows one stack of cards – a complete deck of 52 standard cards plus two jokers. The card images used are from the web site http://www.jfitz.com/cards/ .
Note: Pointing gestures to cards, piles, and places on the table must be made before the Enter key is pressed at the end of a command in English. That is, pointing can be done before or after a command is typed, provided it is done before the Enter key is pressed to begin interpretation of the command.
Individual cards can be moved by dragging them with the mouse while holding the left button down.
Piles of overlapping cards can be moved by shift-left-drag (holding the shift key down and moving the mouse with the left button down). Mnemonic: Think of “shift” meaning “move”. Note: If a pile being moved encounters another pile, those piles are immediately combined into a single pile.
Individual cards can be turned over by ctrl-left-click (left click while holding the control key down). Mnemonic: Think of “controlling” a card as turning it over. A card which has been turned over can continue to be moved by dragging it with the mouse, with or without the control key remaining pressed.
Actions: Commands can request the agent to shuffle, spread out, or stack up piles of cards, or to turn cards or piles of cards face up, face down or over.
References to objects: Individual cards and piles of cards can be referred to
by clicking on them with a mouse (or other pointing device) and using deictic determiners or pronouns in the noun phrase of the command: “this”, “that”, “these”, “those”, “it”, “them”, or the definite determiner “the”;
by using anaphoric determiners or pronouns – the same as the deictic ones or definite ones but without pointing – to refer to a card or pile(s) most recently referred to;
by using quantified definite reference in various ways:
with the unique quantifier determiner “the” (without pointing) and the noun “pile” (i.e., the noun phrase “the pile”) to refer to the only pile of two or more cards on the card table top, assuming there is only one such pile.
with the unique quantifier determiner “the” (without pointing) with the noun “card” (i.e., “the card”) to refer to the only card which is not in a pile with other cards, assuming there is only one such card.
by using a universal quantifier “all”, “each” or “every”. “All” can be modified further with adverbs “separately” or “together”. “Each” and “every” imply “separately”, but “separately” can also be used explicitly with them.
References to places: A place on the "card table" surface can be referred to by clicking on the green surface, or on a card image (which also indicates the card or its pile), and using a deictic adverb "here" or "there". A place can also be referred to anaphorically, by using "there" (without pointing) to refer to the last place that was referred to.
Notice that it is a non-trivial task to resolve references to objects, because pointing to a card may involve reference to the card or to its pile, and because direct manipulation permits piles to be changed between input sentences. Also, it is not trivial to resolve references to places, because pointing to a card may also involve pointing to the place where it lies.
The syntax of commands understood by the card agent is specified by the ASD grammar Cardgram1.grm . A command may consist of one clause or of two clauses conjoined by “and”. The vocabulary of words recognized by the card agent is:
all, and, card, cards, down, each, every, face, here, it, next, now, of, one, out, over, pile, piles, separately, shuffle, spread, stack, stacks, that, the, them, then, there, these, this, those, together, turn, up.
Examples of valid commands are
shuffle the cards
spread them out face down
turn this pile over and stack it here [with pointing to a pile and a place]
turn each of the piles face up
shuffle
each pile and stack it separately face down
shuffle the piles and stack them together here [with pointing to a place]
turn every card over and spread them all out together
turn over this card [with pointing to a card]
spread out this pile [with pointing to a card in a pile]
turn down each of the piles
shuffle all the cards and spread them out face up here [with pointing to a place]
[move a card away from the pile] stack the pile here
The implementation consists mainly of the following Java classes
CardWorld – the driver class
Card, AbstractPlayingCard, and PlayingCard – to represent the cards
CardGroup, CardPile, and PlayingCardDeck – to represent collections of cards
CardTable – to represent the table top and the positions of the cards and piles on it
CardAgent1, CardAgent1Effectors, and Cardgram1Semantics – to represent the card agent, its abilities to operate on the card table, and its ability to understand commands in English related to the Card World. Note: Cardgram1Semantics contains all of the functions that implement semantic action and semantic value fields in Cardgrm1.grm.
CardWorldFrame1 – to implement the window interface between the Card World and the user.
Due to subtleties in the semantics of English, the logic for finding the referent(s) of a noun phrase is the most complicated part of the implementation. That logic is expressed mainly in the instance functions with names beginning findReferents... in the class CardAgent1. Those functions implement a decision network which selects one of the following sixteen different referent types depending on the values of the twenty different syntactic, semantic, and pragmatic decision variables enumerated below.
[The abbreviations Dx, Qx and Ax
are
used in comments to label appropriate parts of the source code
in CardWorld1 (class CardAgent1,
the functions findReferents...). They are also
used
in appropriate parts of the source code for CardWorld1a
(class CardAgent1, function findReferents). Finally, they
are
used to label final nodes in the
decision networks CardDecisionNet1a.grm
and CardDecisionNet1b.grm.]
Deictic
(i.e., with pointing):
D2. the pile containing the card most recently attended to (e.g. "that pile")
D3. each card in the pile containing the card most recently attended to
(e.g. "each of those cards")
D4. all of the cards (together) in the pile containing the card most
recently attended to (e.g. "all those cards", "them"?)
Quantified, universally or uniquely (i.e., all instances of or unique instances of a type):
Q1. each card on the table separately (e.g. "each card", "each of the cards", "every card")
Q2. all of the cards on the table together (e.g. "all [together]", "[all] [of] the cards")
Q3. each pile on the table separately (e.g. "each [pile] [separately]", "every pile",
"each of the piles", "all [of] the piles [separately]")
Q4. all of the piles on the table together (e.g. "all [of] the piles together"
Q5. the only card that the command would sensibly refer to
(e.g. "the card" when there is only one card by itself, the others being in multi-card piles)
Q6. the only pile that the command would sensibly refer to
(e.g. "shuffle it" or "turn the pile over" when there is only one pile containing more than one card, or "turn it over" when there is only one pile on the table, or "turn them over" when all of the cards on the table are in a single pile)
Anaphoric (i.e., backward reference without further pointing):
A1. the last card referred to (e.g. "it", or "that [card]" without pointing)
A2. the last pile referred to (e.g. "it", or "that [pile]" without pointing)
A3. the last cards referred to, separately ("them", "those [cards] [separately]")
A4. the last piles referred to, separately ("them", "those [piles] [separately]")
A5. the last cards referred to, together ("them", "those [cards] [together]")
A6. the last piles referred to, together ("them", "those [piles] [together]")
?1. Is the current clause the first in its sentence?
?2. Has a card been pointed to since the last sentence?
?3. Has the explicit type "card" been used in the current clause?
?4. Has the explicit type "pile" been used in the current clause?
?5. Was the previous type unexpressed, "card", or "pile"?
?6. Is the noun or pronoun in the current clause singular or plural?
?7. Is the quantity of referents expressed in the current clause one or more than one?
?8. Is a deictic pronoun or determiner ("this", "that", "these", or "those") used in the current clause? [Note: Those words may also be used anaphorically, or even as unique quantifiers.]
?9. Is an anaphoric pronoun ("it", "them") used in the current clause? [Note: Those words may also be used deictically, or even as unique quantifiers.]
?10. Has "each", "every", or "separately" been used in the current clause?
?11. Has the adverb "together" been used in the current clause?
?12. Has separate aggregation (originally expressed by "each", "every", or "separately") continued to be used in the most recent preceding clause?
?13. Can the verb in the current command apply to single cards or only to collections of two or more cards?
?14. Was there a single last card referred to before this clause?
?15. Was there a single last pile referred to before this clause?
?16. Were there two or more piles referred to before this clause?
?17. Was the last operation performed on all CARDS on the table (as opposed to all of the piles, or just some of the cards)?
?18. Is there only one pile on the table?
?19. Is there only one multi-card pile on the table?
?20. Is there only one singleton
card on the table?
http://en.wikipedia.org/wiki/Anaphora_%28linguistics%29
http://en.wikipedia.org/wiki/Deixis
http://en.wikipedia.org/wiki/Indexicality