Академический Документы
Профессиональный Документы
Культура Документы
Tom Griffiths
Noam Chomsky
Language
a set (finite or infinite) of sentences, each finite in
length and constructed out of a finite set of
elements
1
0
all sequences
Machines
Computable
Context sensitive
Context free
Regular
Finite
Turing machine
Bounded TM
Push-down automaton
Finite state automaton
Geography
Outline
The argument for innate knowledge
Break
Formal models of learning
Platos problem
Socrates. And at present these notions have just been stirred up in him, as in a dream; but if he were
frequently asked the same questions, in different forms, he would know as well as any one at last?
Meno. I dare say.
Socrates. Without any one teaching him he will recover his knowledge for himself, if he is only asked
questions?
Meno. Yes.
Socrates. And this spontaneous recovery of knowledge in him is recollection?
Meno. True.
Socrates. But if he did not acquire the knowledge in this life, then he must have had and learned it at
some other time?
Meno. Clearly he must.
Socrates. Which must have been the time when he was not a man?
Meno. Yes.
Socrates. And if there have been always true thoughts in him which only need to be awakened into
knowledge by putting questions to him, his soul must have always possessed this knowledge
Modernizing Plato
Q: Why do we know so much?
A: aspects of knowledge and understanding
are innate, part of our biological endowment,
genetically determined, on a par with the
elements of our common nature that cause
us to grow arms and legs rather than wings.
(Chomsky)
Universal grammar
The set of languages that human beings can
learn must be strongly constrained
(in a way that allows the rules and principles of
grammar to be identified from limited input)
Gavagai!
(Quine, 1960)
Possibilities:
Rabbit
Dinner
Undetached rabbit parts
Momentary rabbit-stage
Mass of rabbithood
Temporal cross-section of a
four-dimensional spacetime extension of a rabbit
Other constraints
The child approaches language with an intuitive
understanding of such concepts as physical
object, human intention, volition, causation, goal,
and so on.
Controversy
Are the premises of the argument valid?
Controversy
Are the premises of the argument valid?
There isnt enough evidence to identify these
principles in the data available to children
How should we interpret this?
weak sense: something is innate
strong sense: linguistic nativism
Break
Up next:
Formal models of learning
Golds theorem
identification in the limit
Golds theorem
identification in the limit
Golds game
How do we show that a set of hypotheses
about the structure of language can be
learned using a particular algorithm?
Golds approach: define a game that the
learner plays against an adversary
If the learner can define an algorithm that
wins the game for every hypothesis in the
set, that set is learnable by that algorithm
Game 1
Set of hypotheses:
{1}, {1, 2}, {1, 2, 3}, , {1, 2, 3, 4, 5, 6, 7, 8, 9, 10}
(call these L1, L2, L3, , L10)
Game 2
Set of hypotheses:
{1}, {1, 2}, {1, 2, 3}, , {1, 2, 3, 4, , }
(call these L1, L2, L3, , L)
Game 3
Set of hypotheses:
{2,3,4,5}, {1,3,4,5}, {1,2,4,5}, {1,2,3,5},{1,2,3,4}
(call these L1, L2, L3, )
Game 4
Set of hypotheses:
N-{1}, N-{2}, N-{3}, where N = {1, 2, 3, , }
(call these L1, L2, L3, )
Game 5
Set of hypotheses:
N, N-{1}, N-{2}, N-{3}, where N = {1, 2, 3, , }
(call these L, L1, L2, L3, )
Golds Theorem
Provided with just a text, any set of languages
which contains all finite languages and at least one
infinite language is not identifiable in the limit.
Machines
Computable
Context sensitive
Context free
Regular
Finite
Turing machine
Bounded TM
Push-down automaton
Finite state automaton
Golds Theorem
Provided with just a text, any set of languages which
contains all finite languages and at least one infinite
language is not identifiable in the limit.
None of the classes of (>finite) languages in the
Chomsky hierarchy is identifiable in the limit from just
positive evidence (i.e. sentences in the language).
Learning language requires strong constraints on the
set of possible languages.
Golds theorem
identification in the limit
1%
1(
N 'log | H | + log *
&
)
Classes of languages in the Chomsky hierarchy
are too large to be PAC learnable (including finite)
Golds theorem
identification in the limit
Objectionable assumptions:
What constitutes a language (a set)
Adverserial learning (how texts are generated)
What learners see (a text, samples from D)
What constitutes learning (identification/PAC)
For Thursday
Read Goodman (1955) on grue