CardWorld version 2, Aug 2010 -- Author: James A. Mason
http://www.yorku.ca/jmason/asdindex.htm

CardWorld illustrates use of ASDParser with an ASD grammar (Cardgram1.grm) to
implement an application that understands English in limited ways. It is the
smallest example which I have been able to construct that incorporates syntax,
semantics, and pragmatics all in non-trivial ways.  Thus it allows easy
exploration of how those three components of language understanding interact.
Also, the pragmatic domain of playing cards is rich enough that this model of
language understanding can be extended in many interesting ways,
syntactically, semantically and pragmatically.

PRAGMATIC LIMITATIONS

CardWorld permits a user to interact with images of ordinary playing cards on
top of a card table.  Cards and piles of cards on the table can be manipulated
directly with a mouse or other pointing device, and they can also be
manipulated by giving commands in English to a "card agent" implicit in the
Card World.  A pile of cards consists of cards whose images overlap.  When a
pile of cards has been stacked by the card agent, the images of the cards are
displayed on top of each other, slightly offset, so that only the upper face
of the top card in the pile is visible, but a side and end of the pile are
visible too.  Initially CardWorld shows one stack of cards -- a complete deck
of 52 standard cards plus two jokers.  The card images used are from the web
site http://www.jfitz.com/cards/ .

NOTE: Pointing gestures to cards, piles, and places on the table must be made
BEFORE the Enter key is pressed at the end of a command in English. That is,
pointing can be done before or after a command is typed, provided it is done
before the Enter key is pressed to begin interpretation of the command.

DIRECT MANIPULATIONS

* Individual cards can be moved by dragging them with the mouse while holding
the left button down.

* Piles of overlapping cards can be moved by shift-left-drag (holding the
shift key down and moving the mouse with the left button down).  Mnemonic:
Think of "shift" meaning "move".  Note: If a pile being moved encounters
another pile, those piles are immediately combined into a single pile.

* Individual cards can be turned over by ctrl-left-click (left click while
holding the control key down).  Mnemonic: Think of "controlling" a card as
turning it over.  A card which has been turned over can continue to be moved
by dragging it with the mouse, with or without the control key remaining
pressed.

COMMANDS IN ENGLISH

1. Semantics

* Actions:  Commands can request the agent to shuffle, spread out, or stack up
piles of cards, or to turn cards or piles of cards face up, face down or over.

* References to objects:  Individual cards and piles of cards can be referred
to

   a. by clicking on them with a mouse (or other pointing device) and using
   deictic determiners or pronouns in the noun phrase of the command: "this",
   "that", "these", "those", "it", "them", or the definite determiner "the";

   b. by using anaphoric determiners or pronouns -- the same as the deictic
   ones or definite ones but without pointing -- to refer to a card or pile(s)
   most recently referred to;

   c. by using quantified definite reference in various ways:

      * with the unique quantifier determiner "the" (without pointing) and the
      noun "pile" (i.e., the noun phrase "the pile") to refer to the only pile
      of two or more cards on the card table top, assuming there is only one
      such pile.

      * with the unique quantifier determiner "the" (without pointing) with
      the noun "card" (i.e., "the card") to refer to the only card which is
      not in a pile with other cards, assuming there is only one such card.

      * by using a universally quantifier "all", "each" or "every".  "All" can
      be modified further with adverbs "separately" or "together".  "Each" and
      "every" imply "separately", but "separately" can also be used explicitly
      with them.

* References to places:  A place on the "card table" surface can be referred
to by clicking on the green surface, or on a card image (which also indicates
the card or its pile), and using a deictic adverb "here" or "there".  A place
can also be referred to anaphorically, by using "there" (without pointing) to
refer to the last place that was referred to.

Notice that it is a non-trivial task to resolve references to objects, because
pointing to a card may involve reference to the card or to its pile, and
because direct manipulation permits piles to be changed between input
sentences.  Also, it is not trivial to resolve references to places, because
pointing to a card may also involve pointing to the place where it lies.

2. Syntax

The syntax of commands understood by the card agent is specified by the ASD
grammar Cardgram1.grm .  A command may consist of one clause or of two clauses
conjoined by "and".  The vocabulary of words recognized by the card agent is:

   all, and, card, cards, down, each, every, face, here, it, next, now, of,
   one, out, over, pile, piles, separately, shuffle, spread, stack, stacks,
   that, the, them, then, there, these, this, those, together, turn, up.

Examples of valid commands are

        shuffle the cards
        spread them out face down
        turn this pile over and stack it here
                [with pointing to a pile and a place]
        turn each of the piles face up
        shuffle each pile and stack it separately face down
        shuffle the piles and stack them together here
                [with pointing to a place]
        turn every card over and spread them all out together
        turn over this card [with pointing to a card]
        spread out this pile [with pointing to a card in a pile]
        turn down each of the piles
        shuffle all the cards and spread them out face up here
                [with pointing to a place]
        [move a card away from the pile] stack the pile here



IMPLEMENTATION NOTES

The implementation consists mainly of the following Java classes

* CardWorld -- the driver class

* Card, AbstractPlayingCard, and PlayingCard -- to represent the cards

* CardGroup, CardPile, and PlayingCardDeck -- to represent collections of
cards

* CardTable -- to represent the table top and the positions of the cards and
piles on it

* CardAgent1, CardAgent1Effectors, and Cardgram1Semantics -- to represent the
card agent, its abilities to operate on the card table, and its ability to
understand commands in English related to the Card World. Note:
Cardgram1Semantics contains all of the functions that implement semantic
action and semantic value fields in Cardgrm1.grm.

* CardWorldFrame1 -- to implement the window interface between the Card World
and the user.

Due to subtleties in the semantics of English, the logic for finding the
referent(s) of a noun phrase is the most complicated part of the
implementation.  That logic is expressed mainly in the instance functions with
names beginning "findReferents..." in the class CardAgent1.  Those functions
implement a decision network which selects one of the following sixteen
different object referent types depending on the values of the twenty
different syntactic, semantic, and pragmatic decision variables enumerated
below.


Object referent types:

Deictic (i.e., with pointing):

D1. the card most recently pointed to or moved (e.g. "that card")
D2. the pile containing the card most recently attended to (e.g. "that pile")

D3. each card in the pile containing the card most recently attended to
    (e.g. "each of those cards")

D4. all of the cards (together) in the pile containing the card most
    recently attended to (e.g. "all those cards", "them"?)

Quantified, universally or uniquely (i.e., all instances of or unique
instances of a type):

Q1. each card on the table separately (e.g. "each card",
    "each of the cards", "every card")
Q2. all of the cards on the table together (e.g. "all [together]",
    "[all] [of] the cards")
Q3. each pile on the table separately (e.g. "each [pile] [separately]",
    "every pile", "each of the piles", "all [of] the piles [separately]")
Q4. all of the piles on the table together
    (e.g. "all [of] the piles  together"
Q5. the only card that the command would sensibly refer to
    (e.g. "the card" when there is only one card by itself,
    the others being in multi-card piles)
Q6. the only pile that the command would sensibly refer to
    (e.g. "shuffle it" or "turn the pile over" when there is only one pile
    containing more than one card, or "turn it over" when there is only one
    pile on the table, or "turn them over" when all of the cards on the table
    are in a single pile)

Anaphoric (i.e., backward reference without further pointing):

A1. the last card referred to
    (e.g. "it", or "that [card]" without pointing)
A2. the last pile referred to
    (e.g. "it", or "that [pile]" without pointing)
A3. the last cards referred to, separately
    ("them", "those [cards] [separately]")
A4. the last piles referred to, separately
    ("them", "those [piles] [separately]")
A5. the last cards referred to, together
    ("them", "those [cards] [together]")
A6. the last piles referred to, together
    ("them", "those [piles] [together]")


Object referent decision variables:

?1. Is the current clause the first in its sentence?
?2. Has a card been pointed to since the last sentence?
?3. Has the explicit type "card" been used in the current clause?
?4. Has the explicit type "pile" been used in the current clause?
?5. Was the previous type unexpressed, "card", or "pile"?
?6. Is the noun or pronoun in the current clause singular or plural?
?7. Is the quantity of referents expressed in the current clause one
    or more than one?
?8. Is a deictic pronoun or determiner ("this", "that", "these",
    or "those") used in the current clause?
    [Note: Those words may also be used anaphorically, or even as unique
    quantifiers.]
?9. Is an anaphoric pronoun ("it", "them") used in the current clause?
    [Note: Those words may also be used deictically, or even as unique
    quantifiers.]
?10. Has "each", "every", or "separately" been used in the current clause?
?11. Has the adverb "together" been used in the current clause?
?12. Has separate aggregation (originally expressed by "each", "every", or
     "separately") continued to be used in the most recent preceding clause?
?13. Can the verb in the current command apply to single cards or only to
     collections of two or more cards?
?14. Was there a single last card referred to before this clause?
?15. Was there a single last pile referred to before this clause?
?16. Were there two or more piles referred to before this clause?
?17. Was the last operation performed on all CARDS on the table (as opposed
     to all of the piles, or just some of the cards)?
?18. Is there only one pile on the table?
?19. Is there only one multi-card pile on the table?
?20. Is there only one singleton card on the table?


References:

http://en.wikipedia.org/wiki/Anaphora_%28linguistics%29
http://en.wikipedia.org/wiki/Deixis
http://en.wikipedia.org/wiki/Indexicality





