I did a BSc, and my final project used neural nets (specifically
LRAAMs and SOMs) to recognise source code (specifically the C
language, but parser could be changed). It was a moderate success -
could fairly reliably recognise some constructs two or three levels up
from the base of the parse tree.

Now I want to get an MSc (2 years by pure research) from the same
area. But I'm stumped.

The ultimate goal (say for the PhD) would be to match English with
source code, so that, eg we could have a program that could read
questions on comp.lang.c, and "answer them".

In the meantime I need something to satisfy (a) me (b) my professor
(c) my company (who shall pay the fees - in this company we write a
lot of source code).

A good answer to this question should include URLs of articles
covering work in and around the proposed subject.
Search terms I have used, and might start your search : AI, Neural
Networks, Programming clichés, program understanding, Programmer's
Apprentice, Abstract Syntax Tree, Natural Language Processing.
Time limit for me: September, so for you : August

---

Clarification of Question by jalanb-ga on 15 Jun 2002 16:06 PDT

Re: d2718j-ga 

(1) No - immediate practical application not required, but if it did
have that might make it easier to get them to cough up the fees.
(Comapny is Telecoms infrastructure manufacturer, but we do a lot of
coding).

(2) "The same area" means program understanding, using AI. Neural nets
are the preferred branch of AI because I have invested a lot of effort
there already. And that is also the speciality of the Professor who
shall be supervising this work. But other interesting branches of AI
might be used.

(3) I am employed full-time, and shall be squeezing in hours.

(4) Yes, the MSc is a stepping stone to a PhD, but not really to make
me more marketable. I am not just jumping straight to the PhD because
I simply want to give more time to it. This is primarily for my own
interest, the only reason for the degrees is to give myself concrete
goals.

(5) Advisor - no. Supervisor - yes. I appreciate your comments on the
importance of the advisor, but I ain't got a lot of choice here.

Re: violette-ga

I am interested in programming theory, not database theory.

jalanb,

Thanks for contacting Google Answers!

My recommendation would be to specialize in an emerging field of
theory called swarm intelligence, also sometimes called emergence. 
These fields study the behavior of a number of individuals, very
similar to neural networking theory.  However, the premise of swarm
intelligence could yield a lot more progress in natural language
processing, AI, and some of the fields you're interested in.

Check out:

http://www.everything2.com/index.pl?node_id=1252406
http://www.everything2.com/index.pl?node_id=1047505

There is a gigantic amount of work being done in this field, but not
very many skilled researchers.  You may find, like I have, that this
field represents a significant frontier not only in artificial
intelligence, but in natural language processing, enhancing neural
networks, etc.

If you're interested in this field, check out:

http://www.mkp.com/books_catalog/catalog.asp?ISBN=1-55860-595-9

Related google searches including the term 'Swarm Intelligence' seem
to reveal a significant body of work with regards to natural language
processing utilizing swarm intelligence theory.

Good luck in your search!  My methodology was doing searches on
Everything2 regarding topic matter I was already familiar with (I am a
researcher in a very similar field, though not accredited).

---

Request for Answer Clarification by jalanb-ga on 15 Jun 2002 19:08 PDT

What, specifically, did you turn up that makes you feel swarm
intelligence is suited to program understanding ?

---

Clarification of Answer by arimathea-ga on 16 Jun 2002 21:50 PDT

jalanb,

Thanks for your request for clarification :-)

My reason for believing that swarm intelligence (and its related
fields) has specific application to your request is primarily based
around the belief, in swarm intelligence, of agents which result in a
global pattern emerging.  Since agents are designed to operate without
much prior knowledge (e.g., they are simple), each set of agents could
be designed to understand a specific program structure.  Not only
that, but the ability to have them "dynamically learn" might have very
specific and real applications for the matching of English language to
program structure.  Of course, this is true for artificial
intelligence/neural networks in general - but swarm intelligence has
real value for problems like the 'travelling salesman' problem you
hear about in a lot of neural network research.

Now, admittedly, there would be a lot of work you'd have to do to get
to this point.  But after all, isn't this what an MSc and a PhD is
about?  I'm not advocating that you choose this specialization - it is
a very personal choice, and one that you will no doubt spend a lot of
time thinking about.  However, with your previous work in neural
networks and artificial intelligence, and your desire to concentrate
in the same area, I think it's a great idea.

Your question is a hard one to answer - it's hard to be objective
about something like,  "Tell me what field to go in."

Some of the places I looked are:

http://web.media.mit.edu/~stefanm/generals/reviews_autonomy_formatted.pdf
http://www.pcai.com/web/issues/pcai_15_6_toc.html#Editorial
http://iridia.ulb.ac.be/~ants/ants2000/

From the Genetic Algorithms list:

http://www.aic.nrl.navy.mil/galist/digests/v14n9

Recently, ant algorithms and swarm intelligence systems have been
offered
as a novel computational approach that replaces the traditional
emphasis on
control, preprogramming, and centralization with designs featuring
autonomy,
emergence, and distributed functioning.  These designs are proving
flexible and robust, able to adapt quickly to changing environments
and to continue functioning even when individual elements fail.

Good luck in your search, and my apologies if my answer wasn't what
you were looking for.

Best,
arimathea

(1) Does the work need to have an immediate practical application in
order for our company to be willing to support you?

(2) When you say "the same area", do you just mean program
understanding, or must it be specifically program understanding using
neural nets?

(3) How much work is involved in the MSc program?  Two years of
full-time research?  Or are you employed full time and trying to
squeeze in a few hours here and there?  It could make a big difference
in the scope of the project.

(4) Is the MSc mostly a stepping stone toward the PhD?  If so, why not
go straight for the PhD?  Or are you wanting something out of this MSc
that will make you more marketable for software development jobs?

(5) Do you have an advisor lined up for the program?  If so, have you
asked him/her for topic suggestions?  If not, having the right advisor
is often more important than the topic.  Find an advisor who seems
reasonable and with whom you get along, whose interests overlap with
your own, and who is well-respected in their field.  Find out whether
their current students are happy, and their former students
successful.  And then be flexible about the topic, if necessary.

One area you might look at is database theory.  Collecting data has
become easier and easier during recent years, but the means of
organizing and 'mining' that data have lagged behind.  How can your 
prior research and experience apply to organizing and searching data? 
If you are looking for a lot of source code to please your employer
(as well as a practical application of your knowledge that might
benefit the business) this might be a direction to investigate.  You
would need some sort of a similar storage mechanism in order to
implement any kind of 'natural language' processing anyway, because of
the large vocabulary and ruleset.