From: Samantha Atkins (samantha@objectent.com)
Date: Thu Sep 14 2000 - 01:03:34 MDT
Emlyn O'Regan wrote:
>
> Right on, I support this position 100%. The fruit or side effect of
> increased automation, depending on your point of view, is increased
> redundancy for people. This isn't always true; as some jobs go away because
> of automation, more jobs are created at the "next level up", doing the next
> thing that the machines can't do yet, and that we never tried before because
> they were too hard without the current level of automation as support. But
> to a large extent, automation has a goal of reducing manual work, and often
> succeeds.
I doubt the "we create more jobs one level up" idea or that it
ameliorates the problem much. I doubt we created more programmer jobs
than the total number of factory workers, secretaries, typists, human
number twiddlers and so on displaced by automation. Even if we did
these next level jobs are reguiring progressive higher skills and higher
intelligence. Looking at the bell-shaped curve for human IQ leads me to
the conclusion that a lot of people are not going to qualify for these
next-level jobs. What should happen to them?
>
> Currently I think that more opportunities are being created as previous
> opportunities dissapear. I don't think we are seeing ever rising
> unemployment statistics in the west; it seems that the opposite is currently
> true. That would be what people call an economy, I guess.
>
You are seeing them in Europe more than in the US. In the US people on
welfare, and the chronically unemployed and a few other categories are
not counted. So the statistics are a bit misleading. Also, I wonder
how many jobs in the US are there simply to keep people employed and
really are pretty pointless today. From what I've seen as a consultant
in large corporations, the number of such is not small.
> However, as the machines get smarter (not really happening yet), the jobs
> left for people will trend more and more toward mentally difficult stuff;
> stuff that the machines aren't smart enough to do, and which many people are
> also ill equipped to handle. This is the track toward AI; Drexler talks
> about intelligent engineering systems, if I remember that book 'o' his
> correctly. I think people (or at least most people) will start to become
> entirely unable to fill any useful job at some point.
>
> If this happened in our current economic climate, I wouldn't think much for
> the chances of these redundant masses. I've read some stuff I didn't really
> understand about "democratisation of capital", maybe someone can explain
> this? Maybe the "new economy" has built in mechanisms which will kick in to
> balance out ensuing social problems from mega-redundancy? I'd be grateful if
> someone informed would comment on this.
>
We are in agreement on the problem. I am not sure what to do about it.
We are at an interesting place. There is not quite enough wealth to
both give everyone a good guaranteed income and let them decide what if
anything to do with their time including working for more money if they
have the skills needed. we're arguably not quite there yet. But
closer. And a pretty good argument can be made that without
concentrations of wealth in private hands their is no energy/resources
to enable many types of innovation. Of course there are also
innovations dying in 9-5 jobs taken just to pay the rent that don't
leave much room or energy for the passions and true interests of many
workers.
> Otherwise, we need something new in the way of economic systems. When
> nano-santa comes, if we are set up exactly as we are now, it's going to be a
> bad christmas for a lot of people.
>
It will be worse than bad if most of the energy goes into endless
competition for more material wealth when material wealth is no longer
that difficult or central. It will also likely be bad if
people/groups/nations are bent on destroying or exploiting one another
with those kinds of abilities in their kit.
> >
> > I worry about another problem. Technology is still marching forward. I
> > support this 101% of course. But, as it advances it automates more and
> > more job levels. I predict that within 5-10 years it will automate most
> > of what is called programming today. This is only one example. As we
> > get more powerful computers at lower cost with more AIish abilities more
> > and more jobs will fall to them. We had best be thinking really hard
> > right now about how we plan to have people have a decent life without a
> > regular for pay job. Despite the current rather slanted rosy
> > unemployment statistics in the US, the problem is real and it will
> > eventually hit even many of us.
>
> Hee hee, the old "programmers will be redundant in 5-10 years" line. Haven't
> heard that for a while; supposedly it's been kicking around since at least
> the 70s. It's no truer today than it has ever been.
>
Don't bet on it. What changes between then and now is the raw power of
the hardware. As it becomes more powerful it becomes more tractable to
automate large segments of the work programmers currently do. Of
course, the demand for programming is so huge that it will take a while
before the automation cuts into the need for programmers too much. But
the level of programmer needed will continue to rise also, stranding
many who are now in that profession. Don't believe me? Wait and see.
> It is true that we continue to automate things which used to be done
> manually. So why aren't all the coders on the dole queue? It's because, as
> something is automated, it becomes qualitatively different (easier) to build
> systems based on that something. In terms of possible systems, entirely new
> vistas open up, which although reliant on the newly automated something, are
> not themselves automated, and require lots of people to build them.
>
Actually, having been in the software world myself for 20 years, I
suspect the truth is a bit dimmer than that. Many of the tools I use
haven't evolved hardly at all in all of that time. C lead to C++ but
the tools used to graft C++ have not exactly gone through any major
revolution. C++ itself is quite primitive in many ways. Java?
Interesting things have been done to exploit some of its features but
the language itself is not that powerful and not sufficient for many
types of problems. Any interpreted or semi-interpreted language with
equal or more reflection could be used in most of the contexts that Java
is used. Some of these languages, such as Lisp and Smalltalk, are or
have been much more powerful and advanced in capability, usage, or
development environment than Java, C++, VB and so on are today. Most of
the central abilities in languages were first invented and explored in
Lisp.
We are beginning to address problems of programming in the large but
frankly many of the solutions are giant kludges that are severely
over-hyped and over-sold. I have gotten quite disgruntled with this
industry. We spend more time trying to lock up "intellectual property"
and out-hype the competition than we do actually designing and building
good systems. And fixing our development tools themselves takes a
backseat to even that. I designed and built things in the 80s ( and I
am not unique in this at all) that are as or more advanced than some
parts of the current highly-hyped baseline.
Sorry. Most of that is an aside and off-topic. I needed to rant. But
personally I don't think software development will get significantly
better until something like Open Source (better add Open Design) and
changes in the basis of software business occur. I don't see how the
current model has room to get out of its own way.
> Once upon a time, to write software, you had to write your own OS. Then OSes
> started to appear, and provided basic functionality (loading a program into
> memory, talking to disks, basic screen handling, and all kinds of other
> stuff). Later, programmers spent a lot of time writing (console based) user
> interface code, reading and writing from primitive disk files, all those
> good things. Then databases started to become common, and GUIs became
> common, and other things became the programmer's task. Now people wanted
> nicer printouts, programs that interoperated, pretty gui front ends, and,
> urm, networking! Programmers wrote lots and lots of code by hand, performing
> those tasks; evetually it was again subsumed into the OS. At some point
> Microsoft invented VB, and programmers were redundant forever shortly
> thereafter.
>
GAG! VB is one of the worst blights exemplifying the very problems in
the industry I spoke of above. The very idea of inflicting on millions
of programmers a language environment that has a badly scaled Basic at
its core pretending to be object oriented (laugh) and which is not even
powerful enough to handle full COM components and on top of this has the
IDE, your program and the debugger in one process so any problems crash
the entire mess - well, the whole thing is simply and utterly
infuriating. I can and have designed better programming environments in
my sleep. Microsoft should be shot for gross ineptitude and programmer
abuse. Of course they were never really about making the world better
for software development They are only in it to capture the market and
eliminate the competition. Or at least any good tendencies in its
people are subsumed by that bottom line over and over again.
Putting various things in the OS is not a very bright idea. The things
in the OS should be the minimum that can be handled more efficiently and
cleanly there. A web browser is not an example of such.
> Or that's the hype... oddly enough, there is such a thing as a VB
> programmer, with possibly more positions available than any other
> language/environment. Because the things it makes easy to do are easy to do,
> sure. But now people can imagine even more complex systems, which they would
> not have attempted before, which they can now just accomplish by writing
> lots and lots of code. If they do these things in VB, it's often even
> harder.
>
VB will not allow you to do most of these things. Back when I (shameful
to admit) hacked some VB I had to write anything interesting in C++/DCOM
and then do the GUI in VB and have it call the C++ COM objects. It
works, but is totally proprietary to Microsloth.
> This continues. As far as I can see, the tools have improved incredibly
> beyond what I was using 10 years ago when I started uni. So much more is
> possible; it's astounding. And yet, paradoxically, it is harder than ever to
> be a programmer. The job is getting more and more difficult. Why?
>
You must be in a very different world or you don't see beneath the
wizards and the push-button IDE. The underlying stuff has largely not
improved at all except for the addition of binary components using CORBA
and COM. Even those two are not a great improvement over a
distributed/persistent peer-peer object application environment I
created in 1986-1987. It was unfortunately ahead of its time and
created under the auspices of a company that had no idea what to do with
it.
Our present underlying compiler and linker technology is not much better
than it was then. Linkers have changed almost not at all. DLLs? Came
out in the mid 80s. The productivity tools for programmers leave a lot
to be desired. In the Microsloth world to browse the call graphs of
functions and objects the software entity must first be fully compiled
with a bunch of special purpose flags set. From one component you can't
browse into such details of another one. The information is not unified
into some database you can query about various software artifacts and
their interaction and inter-dependencies. What data that is gathered is
in a Microsoft proprietary format that you cannot use to develop
something more intelligent. Yet Lisp and Smalltalk environment have had
such abilities for the last decade or even two. I wrote such an
information extractor myself for some <gasp> Cobol legacy stuff I got
stuck with once in 1984.
The reason it is getting more difficult is the systems needed are more
sophisticated and the tools and infrastructure have not kept up with
them. It is difficult to pour energy into better programming tools when
that kind of product doesn't pay off so well at the bank to make the VC
happy. It is difficult to build them in house without the sponsoring
management looking like they don't have an eye on the bottom line. So
we race faster and faster armed with inadequate tools and every more
pressing requirements, busily trying to automate everyone else's work
except our own.
How does this tie-in to extropian interests? The future and the tech
we all so crave is built largely on top of software. Keep most software
proprietary and don't invest in better software tools and the future
will be much more stunted than it could/should be.
> Well, because people want to interoperate. Heterogeneous networks,
> applications, software layers, layer upon layer like a crazy croissant,
> building new, more ambitious, more interesting systems than ever before. The
> new economy, with the 'net as the centerpiece, is a mish mash of components
> and chunks and layers and stuff, all trying to talk to each other, all
> idiosyncratic, brittle, cranky code stretched out like a net weaved of scrap
> metal. Making this stuff hang together is a mad job, done at a frenetic
> speed as old technologies die and new technologies are born. There's no
> comfort zone. There's no pause for breath. There's certainly no sign of AI
> taking over any time soon!
>
It is more cranky and brittle than it needs to be in part because of the
problems I mentioned. There are also some real bearish problems in some
of our currently dreamed up systems. Things that will take real R&D
projects to solve and then under the current model would come out as a
bunch of hacked up tools positioned to maximize profit instead of being
shared across the entire industry that needs them so desparately.
> People have been worried about the contrary point of view; that our systems
> are getting so big, so unwieldy, that at some point we cross a failure
> threshold, beyond which we cannot, as a bunch of humans, reliably maintain
> the systems any more. Why would this be true?
>
Without the proper tools and without more and better automation it is
inevitable.
> Well, much of the problem is that, even though basic OS features are
> "automated", even though the user interface (gui), persistent storage
> (databases), communications (networks) are "automated", they actually
> aren't. These things work well now, but they are never quite perfect (some
> people would say they are not even close!). Just ask UI programmers,
> database programmers, OS coders, network engineers. All the hordes of
> people, working to make/maintain the "automated" systems, the stuff that is
> already "done".
>
My greatest expertise is in object persistence. Persistence is far, far
from "automated". Persistence cross-cuts applications and products but
is often done as a series of hacks within a particular project
life-cycle. Or a product is bought that promises to take the worries
all away but actually seriously perverts all application building
thereafter because its needs have to be met for the application to work
at all and its needs are to perturbing to everything else. And the
solution ties the product and the organization often to the solution
provider firmly. At the moment there is not a good persistent
middleware out there that fully meets what is needed. There are various
attempts of greater/lesser goodness. I plan to do large parts of that
problem better and to eventually release a series of Open Source
persistent middleware tools. I am tired of seeing ugly solutions to
this set of problems I know well.
We have automated certain classes of GUIs but the GUI story is far from
complete or adequate. Many projects are now being seriously perverted
to use only a Web Browser as their GUI! It is as if we are stepping
firmly into the past and putting more and more energy into centralized
servers even though we have more power on our desktops than we dared
dream of in the very recent past. We need good massively distributed
peer-peer systems. Not a sleazy 21st century rework of time-sharing
systems.
> Something really different will have to happen to change this picture. Super
> intelligent AI could do it (so could the wish fairies). Genetic programming
> advances might have a shot, but will still require humans to coordinate them
> at some level, especially given that our ambitions will increase by orders
> of magnitude as our abilities increase, to keep us focused on what is just
> out of reach. The job of configuring such magical genetic programmed
> automated systems to reach these new goals will look a lot like, well,
> programming.
>
Sure. Although of a quite different sort. What disturbs me is how
often I am still doing the same tired old tasks in much the same tired
way. There is not often enough time to both meet the current
over-inflated deadline for an underdesigned product and automate as much
of my own process as would satisfy me.
> Sure the techniques will change. Sure the skill set required will change.
> But the basic programming job will remain, and grow wider in my opinion.
> Lots of things will begin to look like programming in the future, which do
> not now. Biotech might get to a point of "automation" where it starts using
> programmers. Through nanotech, even the bricks & mortar world will start to
> become a programming concern.
>
We don't yet do enough with well-defined and trusted components and with
good tools for finding the right components and simulating their
interaction. Much of our code base is still language and OS dependent
and not componentized at all. Most of code is still application
stovepipes with little/no reuse or reuseability. In short, almost no
automation or next-level engineering applied to our own work. It had
better not continue like this.
- samantha
This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 15:30:58 MST