"Memoranda" and "agenda" are both Latin words.  More precisely, they are gerundives.  A gerundive is a verbal adjective.  Perhaps the most famous use of a gerundive is the passive periphrastic phrase Carthago delenda est! (Carthage must be destroyed). 

As with Latin adjectives, gerundives are declined, meaning that their endings change based on number and case.  Memorandum is the neuter singular form of the Latin verb memorare (meaning to remember).  The neuter plural form of memorandum is memoranda.  Similarly, agenda is the neuter plural form of the Latin verb agere (meaning to put into motion).  

Is there a neuter singular form of agenda?  One certainly exists in Latin - agendum.  English speakers, however, have abandoned this singular form even though they continue to use the singular and plural forms of memorandum.  The estimable H.W. Fowler describes the situation as follows:

"Although agenda is a plural word, it is pedantry to object to the common and convenient practice of thus treating it as a singular one.  If a singular is needed for one item of the agenda there seems no escape from the rather cumbrous phrase; agendum is pedantic and agend obsolete."

Fowler's Modern English Usage (Second Ed.)  The Court of Appeal, however, has boldly ignored Fowler's charge of pedantry:

"More importantly, however, the restriction on the agendum at a special meeting is not, as the CSEA contends, a species of the constitutional limitations on initiatives or legislative appropriation that limits them to a single subject."

Hard v. California State Employees Association, 96 Cal. App. 4th 708, 715 (2002) (footnote omitted).