From: Mike Linksvayer (ml@gondwanaland.com)
Date: Sun Mar 31 2002 - 21:49:35 MST
On Sun, 2002-03-31 at 01:13, Samantha Atkins wrote:
> Mike Linksvayer wrote:
> > OTOH I know a number of good brains that have gone back to school, taken
> > up working on their own projects and/or traveling. These sorts of
>
> Great if you can afford it. Many of us can't.
Anyone _can_ make lifestyle/expense judgements to work on their dreams.
Most don't want to badly enough. Including me most of the time,
unfortunately.
> > things cause financial stress, but in the long run may better serve
> > creativity and innovation than a longer run at theoretically innovative
> > startups rapidly morphing into overweight bureaucracies dominated by
> > politics.
>
> I doubt it but time will tell.
Another positive spin on the .com fallout from
http://www.economist.com/science/tq/displayStory.cfm?story_id=1020789
Another factor that may boost the prospects for AI is the demise of
the dotcoms. Investors are now looking for firms using clever
technology, rather than just a clever business model, to
differentiate themselves. In particular, the problem of information
overload, exacerbated by the growth of e-mail and the explosion in
the number of web pages, means there are plenty of opportunities for
new technologies to help filter and categorise information—classic
AI problems. That may mean that artificial-intelligence
start-ups—thin on the ground since the early 1980s—will start to
emerge, provided they can harness the technology to do something
useful. But if they can, there will be no shortage of buzzwords for
the marketing department.
> > All of your critiques were insightful, but I think overly discount the
> > bright side: sure nothing I mentioned is new, but widespread
> > understanding and adoption is, and in my (again, rather limited)
> > experience each accrues productivity gains and frees one to concentrate
> > on higher level solutions that are just as hard as the lower level ones
> > we don't have to recreate. I would hate to go back to even 1995.
>
> As a long term working software engineer I quite honestly have
> not seen much widespread understand and adoption of even very
> old technology. Nor have I seen much freeing to concentrate on
> higher level solutions. Due to a paucity of automation of our
> own activities, much of practical software engineering is still
> rather laborious and low-level. I am deeply unhappy with this
> situation and will do what I can to improve it.
Software engineering is certainly laborious, and probably will be so
long as wetware is involved. People are working in roughly the same
manner, but AFAICT, the solutions they're working on tend to be higher
level. No longer does a programmer need to re-implement many
fundamental algorithms for each new project. No longer does a
programmer need to re-implement basic services for each new project. Or
in either case copy or pay big bux for, inevitably creating a
maintenance/licensing/dependency nightmare. Standard libraries, free
extensions and application servers provide all of these on every popular
platform, meaning _nearly every programmer today has access_ and can
concentrate more on solving domain problems. Certainly all of this has
been brewing for much longer than a decade, but I'm glad it's coming to
fruition.
I gather now that you won't see progress until the the software
engineering process itself becomes higher level, not just the solutions
software engineers work on. I'm all for it, but I appreciate the mere
progress that has occurred.
> Personally I would happily go back to around 1995 or earlier to
> attempt to change some wrong or less than optimal directions
> taken by myself and the software industry. But hindsight is
> 20-20. We just should not congratulate ourselves on where we
> are today. It is not that significantly better than 1995 or
> even 1985 for that matter. Things I understood then are even
> today understood by only a relatively few and are not in
> practice. Also many things I saw as software weaknesses and
> possible long-term problems are now commonplace and magnified
> far beyond what I dreamed could occur then. But like many of us
> I had a family to support, bills to pay and was busy "making a
> living". And, to tell the truth, I was too much into building
> real systems to stay too much more theoretical or academic.
> Sometimes I feel guilty for not finding a way to push harder
> earlier and especially for being largely out of the loop from
> around 1991-1995 due to personal life complications.
I'd like to go back in time also, but we'll have to disagree about
whether we're significantly better off now than in 1995. Perhaps my
mind will have changed by 2009. Hopefully it will have due to the next
7 years being golden. (I started programming professionally in 1995,
though I've been writing the odd bit of code for much longer.)
If you care to take the time to tell, I'm really interested in the
long-term proplems you forsaw that are now commonplace/magnified. What
would you have done differently (not personally, in terms of the way
things developed), what needs to be done to remove these problems going
forward, and what preventable long term technical problems do you see
brewing now? (I'm well aware of relevant legal/political problems, and
am already deeply concerned about those.)
> > Certainly just as much code is rewritten as ever, even within companies,
> > but if you consider the broad and occasionally deep libraries that are
> > now available for any popular environment you've got massive code reuse
> > happening.
>
> Which do you have in mind? The Java libraries? They cover a
> fair amount but I wouldn't consider them deep or
> well-implemented overall. Their IO library is particularly
> odious.
java[x].* is perhaps the most widely known/used, but by no means the
only example. Libraries that cover roughly the same ground are now an
absolute must for language adoption. You probably know about java.nio
available in 1.4. Yes, languages with excellent libraries have existed
for some time. But they weren't widely used. How many programmer years
have been wasted re-implementing hashtables and linked lists in C over
the years? The libraries for Java, C#, C++ (after a fashion), Perl,
Python, Ruby, etc. go well beyond simple data structures. Beyond those
languages' "standard" libraries, there's a huge amount of easily
reusable code available, much of it open source. Sometimes not easy to
find (with the exception of Perl's CPAN), but Google [groups],
sourceforge and freshmeat are pretty helpful.
> Good libraries have been around for particular
> languages and environments for some time. Making these language
> and environment agnostic has been largely dreamed of but too
> seldom acheived. Many useful components also exist deeply
> ensnared within various applications, often redundantly so.
> Admittedly .NET is a step toward this as are COM, CORBA and ILU.
>
> A big problem on reuse is simply knowing what is available and
> what its requirements and characteristics are in sufficient
> detail to use it or extend it. It would especially be useful if
> this was known programmatically. But such categorization of
> algorithms, functions, classes, methods and components is a
> science unto itself and one not yet mastered much less widely
> implemented.
>
> In the meantime we take a lot of relatively inflexible weak
> languages as the current manna from heaven along with an
> over-hyped app server framework or two and ignore many of the
> most significant problems.
>
> Perhaps you have to be in the midst of this world for some time
> to see what I have seen. Many who have come into the field more
> recently don't know a lot about what went before, the problems
> already identified and solutions that the commerical software
> industry has largely forgotten or never taken out of the lab.
Don't want to argue with any of the above wisdom, worth repeating.
> First of all, I would choose the most powerful language to date
> for AI and for creating new languages and tools, Lisp. It is an
> ancient tongue but one full of power. In any event highly
> reflective language and coding environment is an absolute must
> for self-evolving software and for high human programmer
> productivity. Software library technology needs to be further
> developed and automated. Distributed and persistent objects,
> procedures and data must be as seamlessly incorporated as
> possible and practical. Intefaces to data storage and
> intercommunication facilities in a uniform manner with fairly
> uniform adaptors to higher level constructs and different client
> languages would be very helpful. Some means to make use of code
> already existent in other languages must be developed and
> deployed that is far more natural than COM/CORBA/RMI. A
> low-level language undergirthing all higher level languages is
> workable (as in .NET) if (unlike in .NET) it is flexible enough
> to support all useful fundamental software constructs known to
> date with room to expand in the future. I would build tools
> that automate as much of the detail as is practical without
> losing understanding of the implementation characteristics.
> Excellent debuggers and profilers are a must. Incorporation of
> design methodology tools and the development of much more
> powerful and broad tools would help quite a bit. These tools
> should be ubiquously available without paying through the nose.
> Tools to reverse-engineer existing code and analyze it would
> be very useful. The industry could use some new ways to view
> and understand software systems, perhaps through graphical and
> other sensory representations. In any case better
> human-computer interfaces and more augmentation of human
> thinking, creativity and communication would help tremendously.
All valuable ideas, though of course none are new. They all need robust
implementation and wide adoption.
> If I was running things :-) I would funnel some energy into new
> types of operating systems like the one I heard of that has a
> kernel (Synthesis Kernel - Massalin 1988) that performs run-time
> code synthesis, fine grain scheduling and lock free optimistic
> synchronization, Also I think we have upcoming challenges in
> fine-grain concurrency that we had best get busy designing
> effective languages and operating systems for now.
According to http://www.cs.arizona.edu/people/bridges/os/research.html
there are a number of Synthesis-inspired research OSes worked on fairly
recently, though it's hard to tell whether any of the projects live. If
I were "running things" I'd also encourage OS research, but I wouldn't
touch it with a small team/few years' funding. Little chance any new OS
will be used by many people, and even then the incubation period is
probably best measured in 5 year chunks.
The only "new" OS I'm excited about right now is EROS, largely because I
think pushing capability security (and persistence) is necessary and
because I think EROS has a possibility of attracting developer critical
mass. Anyone following along who wonders what I'm talking about, the
following are pretty gentle introductions:
http://www.eros-os.org/essays/capintro.html
http://www.eros-os.org/essays/reliability/paper.html
As you read you'll probably be struck several times with thoughts along
the lines of "My SI/upload/whatever _needs_ this."
> It probably depends on what you do and do not mean by security.
> Capability based security is pretty much required to get much
> beyond today's software mess.
Yes, see above. Today's software mess will live for a very long time so
needs to be migrated to capability systems or vice versa. There was a
fascinating thread "Saving the UNIX API" on cap-talk
http://www.eros-os.org/pipermail/cap-talk/2002-February/thread.html
mid-February concerning this. I'm also interested in, um, social
security, i.e., how do you decide what to trust. Raph Levien's
thesis-in-progress http://www.levien.com/thesis/thesis.ps on
attack-resistant trust metrics makes for some interesting reading.
Mike Linksvayer
http://gondwanaland.com/ml/
This archive was generated by hypermail 2.1.5 : Sat Nov 02 2002 - 09:13:10 MST