[Copied Bcc to Tom K]
Philippe wrote:
> As you well know, we are working on the port to OS X. As you're probably
> also aware, we find this project more challenging than expected.
I appreciate that coming 'cold' to the task of porting as complex a
system as poplog (especially as it has many unconventional featurs and
is self-bootstrapping) is very challenging. In the past nobody was ever
expected to do it without working closely with the original developers.
Unfortunately they are not now available (in particular John Gibson
retired on account of ill-health, including failing eyesight, a few
years ago).
> First, here's the status of our project. We compile and link since a long
> time ago. We are left with runtime related errors that we address one after
> the other. Most of the problems we have are related to the assembly used.
> For that part, we based our code on the AIX version of poplog. Provided the
> nature of some assembly errors that were found in the assembly, our level of
> trust of the AIX port is highly limited. This obviously makes the project
> way more difficult. So, the first question is the following:
>
> - How stable is the AIX version of poplog? To your knowledge, is this
> version used in production anywhere, or was this a preliminary port that was
> never fully tested?
This port must have been done for ISL before they were bought by SPSS
and I have never met anyone who actually used it. However, looking at
the a*.s files it seems that the core work was done by John Gibson, who
was one of the most thorough programmers I have ever met and was the
chief architect of Poplog.
When ISL were bought by SPSS and it was agreed that all versions of
Poplog should equally belong to Sussex University and SPSS, to do with
what they wished, there was no version of AIX poplog with binaries in
the sources at Sussex. This is presumably because nobody there actually
had a machine on which to build and run it.
I don't know whether ISL had such a machine, or whether they were paid
by a customer to do the port and ran it only on the customer's machine.
It may have been a customer who wanted Clementine to run on AIX.
I shall copy this message to Tom Khabaza who may know the answer, or may
be able to find out the answer.
> As it stands, we experience many problems with r12 (the stack-pointer).
I believe the main reason that Poplog makes so much use of assembler
instead of C is that C does not allow the use of global register
variables, whereas the stack-pointer needs to be both global and a
register variable.
Presumably that is potential cause of many problems.
> We
> fixed the issues related to brk and sbrk. We ran into some assembly level
> bugs that we fixed. Some bugs were fixed generically (in gen_proc.s) and
> some others required a post-cross-compilation changes since they could not
> get fixed with a generic change (the generic change generated some troubles
> in other areas). That increased our suspicions towards the quality of the
> assembly being generated.
>
> As we go though these changes, we can't see the forrest from the tree. We're
> always fixing the most immediate trouble without any way of knowing what is
> behind. That is a serious problem for us and our lack of tools is a surprise
> to me for such a mature product like poplog which was ported to so many
> other platforms.
>
> My expectations would have called for a port utility that would test for all
> the different aspect of a port prior to even trying to run newpop11.
The bulk of poplog was developed piecemeal by people who were learning
a lot about software engineering while developing the system: none of
them was explicitly trained as a software engineer, as far as I recall.
None of them had previously worked on large software engineering
projects. Many were AI graduates from Sussex University.
I remember that at one point when I was still at Sussex we decided that
it was important to have a large collection of tests to be run whenever
the system was rebuilt, to ensure that that new changes did not stop old
things working. Producing such a collection of tests was extremely
difficult because Poplog could be run in so many ways, and many of them
depended on interactions with a user (e.g. the editor, the incremental
compiler, and the X window facilities, which introduced asynchronous
processes within poplog).
I don't know what happened to the partial collection of tests produced
at that time (late 1980s I think): I have looked for it without success.
They were tests to be run on a working system, so might not have been
help for a nearly working system.
One of the things we discovered was that, because poplog was used to
build itself, the process of re-building, including compiling many
different kinds of source files to build all the standard saved images
used so much of poplog that most bugs were detected by that process.
In addition the developers were using the poplog editor for all their
work, and that provided more tests.
So the developers were constantly using almost all the components that
end-users were using. For a system whose end-users have very different
needs from the system developers, testing by developers in their normal
work could not be so deep.
A further factor was that poplog was constantly being used for teaching
and research in the same place as the developmeent work. So there was
much additional testing going on all the time.
However, this meant that a newly ported version that was not being used
by the developers could include undetected bugs. We have found a few
such bugs recently in linux poplog, which was never used by the
developers, including the recently discussed external symbols bug
(now fixed in linux poplog).
> For
> example, a pop11 file that would manipulate the heap, stack allocations,
> list creations and list referrals and else with significant feedback at each
> step to validate the purity of the assembly that was generated by the port.
> This would work a bit the same way as when one writes a java virtual
> machine. One does not test a java virtual machine just by running java
> applets and hoping for the best, but one can run a java virtual machine
> validator that runs all aspects of a virtual machine while validating the
> result of each type of operations. As it stands, we're stuck at running
> newpop11 and hoping for the best, and that is not the most productive way of
> performing a port. We need a port validation utility. So, here's my second
> question to the group.
I agree that that would be a very useful tool. I think Robin Popplestone
(who died recently) developed a partial tester at Univ. of Massachusetts
at Amherst. But I don't know how far it got.
> - Is there a port validation tool that allows us to check for the purity of
> the assembly being generated by the cross-compilation? If not, we'll need to
> write one. Anyone could give us an idea on everything that we need to check
> for all the assembly that can be generated to be checked successfully?
> Already, through a small pop11 file that we wrote and ran we saw some
> problems in regards to assignation of values to variables (I told you we had
> some trouble with the stack-pointer r12), but such a tool would be of
> significant use to ensure the purity of ALL the assembly generated. Then,
> running newpop11 would become almost trivial.
If nobody has already done this, I guess the obvious place to start
would be to work through
1. the specifications of the poplog virtual machine instructions in
REF VMCODE, producing a set of tests for each such instructions.
followed by (in no particular order)
2. the specifications of all the built in sub-routines available to
users and the system, including mathematical operations, operations on
standard data-structures (lists, vectors, classes, procedures,
processes, etc.), and control operations (e.g. chain and related
mechanisms, the process mechanisms).
3. the external interface (e.g. that would have detected the things
that have recently caused problems when I tried using new features of
the popvision library in linux).
4. All the operating system interfaces, including asynchronous event
handlers, pipes, files, sockets, timers, permissions, etc.
5. The compilation process, including macros, syntax-words, tracing,
etc. (This may be redundant if all the previous tests work.)
No doubt other things I have forgotten.
I think this would be a wortwhile project, but quite a big one.
>
> I appreciate your feedback. I am sorry about all the troubles we're having
> with that port (and all the difficult questions we ask), but we're not much
> of quiters here. This is a difficult project that we will complete whatever
> the cost.
I have never worked on a port or developed any of the lowest level
mechanisms in Poplog, so I regret that I cannot be of more help.
When you get to a working system I can provide tests that will use many
parts of the system including the X window interface, objectclass, and
asynchronous event handling (i.e. the programs partially demonstrated in
these movie demos:
http://www.cs.bham.ac.uk/research/poplog/figs/simagent/
Aaron
|