numerals, and language with an infinite lexicon

Here is an idea that I've seen in several places: "it is plain that if a language is to be learnable, the number of basic significant elements (words) has to be finite" (Tim Crane, The Mechanical Mind, 2nd ed. p.140).  (Is the locus classicus for this Davidson's "Theories of Meaning and Learnable Languages?")

But how does this square with numerals (words for numbers)?  There are an infinite number of numerals.  And arithmetical language is learnable. 

So how could/should Crane, Davidson, et al. deal with this apparent counterexample?  I'm not sure... perhaps (despite appearences) numerals are  not themselves genuine words; rather only the digits are genuine words, and the higher numerals are complex supra-word symbols.