Personal computing history (part 8) 6 Jan 2010, 9:12 pm

So, I'm working in the exciting world of embedded programming. Things like "shoot a laser beam at an object and figure out how far away the object is". Or "control this motor, and turn on the laser when it is not pointing at a corner on the attached mirror wheel".

While doing this, I am introduced to the world of PIC microcontrollers. I find this very interesting, because of a project that had some notoriaty at the time: the iPic web server.

Besides being a neat hardware hack, this seems (at the time) like a good way to learn TCP/IP. After all, if you can fit a webserver into a 1K instruction microcontroller, how hard can it be?

So, I start playing with the development tools. And soon find that I don't like them (surprise, surprise...) What do I do?

Well, write my own of course. I'd written VIM, so why not an interpreter for a much simpler processor? Well, I didn't realize it at the time, but there were enough differences in chips and limitations to the compiler I was using that I needed to have a family of interpreters, one for each processor...

I'm back... 27 Dec 2009, 2:48 am

OK, I'm back.

It was a lovely trip.

And I had a lovely bout of bronchitis.

More later...

I'm Off! 29 Nov 2009, 10:54 pm

But not in the way most people think :)

Off on vacation, You may not see anything from me for a few weeks.

Upcoming events... 25 Nov 2009, 6:35 pm

Tomorrow, I run a 10K in the morning, and have Thanksgiving dinner in the afternoon.

On Monday, I board a plane for a warm, sunny destination.

And, as a followup to Apple Update, I have bought a PMP -- a Viewsonic VPD400. 800x480 4.3" screen, plenty of CPU horsepower, lots of codecs. Only 8GB of flash, but it takes MicroSCHC cards so I can carry all the video I can afford...

I would have preferred the VPD500 (5" screen, 16GB of flash), but while it has been announced it is not yet available.

And, for the curious, it's a lot like the Onda 535 (Yah, that's the 545, not 535. I couldn't find a link to the 535, but they are the same in all respects except a 5" screen on the 545). I considered the Onda545 (or even the 797, with a 7" screen!) but
    1) I'd prefer a brand name (and vendor) that was better known in the USA
    2) I was not sure that mp4nation could deliver in time.

And if you are interested in other PMP info, check out PMP Today.

A day that will live in infamy... 23 Nov 2009, 6:59 pm

Happy Birthday to me,
Happy Birthday to me,
Happy Birthday dear me...
Happy Birthday to me.

What, you thought the subject has some other meaning?

Not gone and Not forgotten! 19 Nov 2009, 9:23 pm

This blog, that is.

Right now, it's crunch time at work and I'm getting ready to go on vacation.

No spare time to do anything, including writing blog entries (besides this hastily typed entry).

Sorry, I'll try to make occasional weird comments just to let you know I'm still around...

Personal computing history (part 7) 13 Nov 2009, 8:13 pm

Despite what we could do with virus scan/removal, the company I worked for (names withheld to protect the guilty^H^H^H^H^H^Hinnocent^H^H^H^H^H^H^H^H^H^H^H^Hno one in particular) was losing it's customer base. We did not have a standout "Windows" program, and the business shrank.

Eventually, it folded.

I took a temporary job doing support work (that stretched to a year and a half, I'm lucky that I got out of there without it being permanent).

There was the "dot-com" boom, where I put in my time with a web startup (they were doing tech support via web interface, and wanted a virus scan component). They wound up with most of the rest of them...

Looking around, I remembered what a friend of mine had told me years ago -- that my skills were a good match for embedded programming. So I went after jobs there.

He was right -- while I had to get "up to speed" on the particular embedded controllers, my experience in assembly language and programming in C right down to the bare metal of the computer was just what they were looking for.

I settled in for the long haul...

Personal computing history (part 6) 9 Nov 2009, 9:22 pm

And, after a while, DOS was no longer king. Windows ruled.

And with Windows came Microsoft's Word for Windows. Version 6.

And then came macro viruses...

From the anti-virus vendors came much shouting "How do we parse these files? What is in these files?"

And from Microsoft came the grudging reply "We cannot tell you."

Not "Will not". "Cannot".

You see, back then Word files (and Excel files) were saved C++ object streams. To save a WORD file, you had C++ write the associated objects to disk. To read it back, you have C++ read the associated objects from disk.

So nobody had "designed" the file. It was just C++ objects.

If you use the right C++ compiler and objects, everything just works. Use something else...

So, the best that Microsoft (again grudgingly) could do was to provide a "reference" implementation of C++ objects to read/write an appropriate OLE file, and some documentation on the internal WORD objects (complete with bugs...)

So, what do you do with this? Well, the anti-virus vendors split into 2 camps:
      1) Use the reference implementation as the basis of the scanner
      2) Use the reference implementation to figure out the actual binary contents of a .DOC file.

The company I worked for went for option #2. I had much "fun" (for strange values of "fun") translating object streams into actual binary data.

But in the end, we knew what was in a Word file (or Excel file or...). Down to the bit. And could parse a Word file in ANY language...

Personal computing history (part 5) 4 Nov 2009, 9:17 pm

So, I now have a pair of tools:
     VIMas a debugger
and
     CODEGEN as a disassembler.

What do I do with them?

Well, around this time, the company I was working for became involved with the first computer viruses.

And I had tools that would let me simulate running any software, and rip apart programs with ease.

Of course, some viruses tried to make it hard. They didn't want to be traced, or encrypted themselves, or even kept the parts of themselves that were not currently executing encrypted (i.e. decrypt the currently required subroutine before executing it, and re-encrypt it when done. This was the "Whale" virus). None of it mattered. One or two virses (like the "Whale" virus) made me create a "tweaked" version of my tools, but that was hardly even a speed bump.

And so I spent many years disassembling viruses and writing anti-virus software...

Personal computing history (part 4) 2 Nov 2009, 9:34 pm

As I mentioned last time, the debugger I wrote has 2 instruction decoders. One of which creates both instruction string and executable information.

This turned out to be VERY useful.

Why?

Well, after a while, I also wanted to disassemble programs, not just debug them.

And I had an engine that allowed me to decode instructions and execute them. So I could know not only that a DOS call or BIOS was being made, I could determine what function was being asked for.

Add an algorithm to separate code and data, and you have a very powerful disassembler...

Personal computing history (part 3) 28 Oct 2009, 8:35 pm

The one problem with interpreters (like VIM) is that they are slow.

In fact, VIM used about 400 instructions to interpret a single instruction. So your program ran 400 times slower than normal.

The early PCs and clones were running 8088 chips at 4.77Mhz. Since they weren't all that efficent (not like the single clock instructions of modern chips), you got less than 1 million instructions per second (MIPS). So, at it's fastest, your 1MIPS program was running at
          1000000 / 400 = 2500 instructions per second

Yah, it was slow.

This meant that all sorts of tricks were used to get any little bit of speed out of it that you could.

One of the biggest tasks of the interpreter was instruction decoding. Taking a byte stream like
          A3 34 12
and determinging that this was the instruction
          MOV [1234],AX

If you work with this for a while, you realize that you are much better off if you have two versions of the instruction decoder:
          1) produces readable strings and computer instructions
          2) just produces computer instructions

Most of the interpreters processing will be internal. You are not (usually) going to be watching every instruction. If you don't produce strings for the user (case 2), it is much faster.

If you are going to be producing strings for the user (for example, tracing a critical code section), you are not going to care (in general) how fast the routine is (the user can't keep up with the computer).

So, you trade code space for speed. A common trade-off.

You just have to be VERY SURE that both decoders are "in sync" so you can switch between them without anything breaking (or interpreting instructions incorrectly...)

Personal computing history (part 2) 26 Oct 2009, 8:28 pm

So, why was this such a big deal?

Well, the best debugger is an interpreter. VIM stands for "VIrtual Machine", and is an interpreter for 8086 machine code.

With it's own memory space, so changing something important (like trashing the interrupt vector table) wouldn't crash the real machine.

Simulated periphirals (like interrupt controllers) and access to the actual machine hardware (keyboard, disk drive) so that it could do real-world simulations (like booting and running a different operating system the the one running VIM). And the ability to turn hardware access off, so you could protect your system from malicious software.

Breakpoint on data access, so you can see how a stray memory pointer is trashing your data.

Breakpoints were setup so that if executing an instruction would cause a breakpoint, the break occurred before that instruction executed (for example, if you wanted to see how memory location 1234 was being overwritten, you could set a breakpoint on 1234. Then, if executing the instruction

MOV BYTE PTR [SI],AL

would write to location 1234, VIM will stop before the instruction executed and trashed memory.)

VIM will also remember the last 2047 instructions executed, so you can figure out just how you got to this point.

Also, save/restore your debugging session to disk. So you can go home for the night and pick up where you left off in the morning. Or play "what if" on code execution, and go back to a known starting state any number of times.

So, I now had a tool that would let me do all sorts of neat stuff. And one that would become the basis for much of my work in the future (though I didn't know it at time time).

Personal computing history (part 1) 23 Oct 2009, 8:11 pm

A long time ago, I wrote a lot of assembly language. I got very good at it.

Some years later, I was working for a company that did educational software. Back in the mid-80's a lot of PC software was copy protected. This was a major pain for us, as we had a $2000 program (back in the 80's this was even more of a painful cost than it would be now...) that broke when used on a faster (back then, 80286) machine.

So, my boss, knowing how good I was a low-level stuff, asked me to break the copy protection. No DMCA back then, I was free as a bird...

So, I started digging into the program with DEBUG. And taking notes...

After about 10 pages of scatterd notes with cryptic values written on them, I though to myself "Why am I doing this? I have a computer!" and so VIM was born. A month of effort got the basics working, and the copy protection was toast.

...to be continued...

Disk Rot... 21 Oct 2009, 8:26 pm

Well, despite my warnings to others about software vanishing and keeping "old bits" around, I find that I've been bitten...

I'm doing some cleaning in the computer room (getting rid of old junk), and looked in a CD wallet that contains archives of my development work. The disks are succumbing to disk rot -- when I opened the sleeves in the wallet, they took the aluminum layer with them. Oops...

I think this was an interaction between that particular brand of CD and the plastic of the disk sleeve, since other CDs in the same wallet are fine.

Fortunately, I made multiple copies. On CD's from different companies. Stored in different locations and methods (CD wallets vs jewel cases).

I haven't lost anything, but that's just because I'm paranoid about such things. But not everyone is...

Back to work... 19 Oct 2009, 8:02 pm

no, not you. Me.

About a week and a half ago I received an email from someone who sent me the source to Intel's AEDIT (of which SEE, the editor that came with DeSmet-C, is a clone). He is setting up a website about SEE, and both wanted me to have the AEDIT source (written in PL/M) and to have me "tidy-up" the DeSmet C pages in advance of being spidered by search engines for the new page.

So, this finally got me to get off my duff and post the DeSmet 2.51 C88 reconstruction I had done.

Though, if anyone wants to make a better DeSmet C page, I won't say no...

Also, I've started back in on the 2.51 GEN reconstruction I had started and not finished. This, as I anticipated, is more complex than the C88 reconstruction, as GEN had undergone more changes, both to support "large case" (i.e. large memory model) and to support direct generation of object code (no intermediate assembly language file).

I'm between 1/3 and 1/2 finsished, and plowing through. It's a bit worrysome, though, that I have several functions that I have yet to discover names for. This is probably due to routines that were subsumed by other functions, but it will mean that I have to "guess" names and code structure, which will make the reconstruction less useable. Hopefully I'll find the equivalent later code...

It's loose! 12 Oct 2009, 8:54 pm

The reconstructed DeSmet C v2.51 source, that is.

It only took me from January 2009 to now to do it...

Easter Eggs, part 2... 9 Oct 2009, 7:30 pm

Easter Eggs can also be in some surprising places. Like computer chips...

Known as Chip Art, these are microscopic doodles put in chip for fun (or copyright purposes, as described in the above wikipedia article).

Some of it is crude, some wonderfully detailed, all of it fascinating.

Check out Molecular Expressions and Chipworks and see if you really know what's inside your computer...

Easter Eggs! 8 Oct 2009, 8:45 pm

OK, not the kind you eat. It's the wrong time of year for that.

But, hidden "goodies" in film, software, TV, or whatnot are called easter eggs.

A common example is a hidden menu selection on a DVD. If you find the right button combination, you are taken to the "hidden" content (commonly a making-of feature or some such).

Another common easter egg is a hidden joke or reference. For example, in LOST, "the numbers" often show up in unexpected places, such as the numbers on police cars in a parking lot.

If you're interested in such things, check out The Easter Egg Archive.

Now with more Fiber! 6 Oct 2009, 7:24 pm

Optical fiber, that is...

It's finally available in our area, so my wife and I have had Verizon FIOS installed.

Even with the "better" package, we have POTS, internet at 25MBPS down / 15MBPS up, and television, with lots of HD channels.

Pretty, and wonderful waste of time...

Down and dirty in unix... 30 Sep 2009, 8:27 pm

Long ago, I used assembly language for the PC (in DOS) every day.

These days, working with microcontrollers, I only dip into assembly when debugging or writing initilization code.

But assembly is far from dead. Even for the PC.

Even for OSes like UNIX.

Yup. While Unix was written in C to be portable, you can get "down and dirty" if you want.

The interface is via (for a PC-based system) INT 80h, which also happens to be the website which teaches an introduction to assembly and UNIX on the PC.

Not forgotten... 28 Sep 2009, 8:43 pm

You were excited. Waiting with baited breath. The new, must-have video game was finally here!

But that was years ago, and that new, must-have video game is long forgotten. The people who wrote it may have some memories, but the company that published it (and owns the copyright) is long gone.

It's call abandonware. And while someone, somewhere owns the copyright (it's not really abandoned), they don't care enough to either sell it or make it available.

So sites like abandonia exist, keeping the files alive.

Yes, it's technically illegal. But if the owners don't care enough to actually sell it to you or even tell you to stop distributing it, why should you care?

Superheroes: where to find them... 25 Sep 2009, 6:03 pm

Since you already have the store for Superhero Stuff, keeping an eye on the store would probably be a good place to start.

Or just check out the World Superhero Registry...

For the Superhero in you... 23 Sep 2009, 8:50 pm

When you read comics (admit it, you did!), didn't you ever wonder where all the neat toys came from?

Costumes, masks, gadgets, robots -- everything you needed to combat evil.

Sure, the Superhero is a genius, but who has the time to do it all?

Well, it turns out that Brooklyn Superhero Supply is the source for all your heroic needs.

Check it out, and maybe you can meet &ltinsert your favorite hero or heroine here&gt and invite them out for coffee...

Instead of reality... 21 Sep 2009, 8:57 pm

how about a little fantasy?

Steampunk rayguns from Weta.

They're pretty...

Daily routine, Part 1... 14 Sep 2009, 8:23 pm

Like most people, I have a number of websites I visit regularly.

Some are humorous. Some are news. Some are educational. Some are just plain weird. Some are beautiful. Some are guilty pleasures.

And some fall into more than one category.

Like Astronomy picture of the day.

Often, I get to see a beautiful picture of some marvel of the universe. Othertimes, I get a lesson on science and how it relates to the world. And sometimes I try to see if I'm as smart as a (hypothetical) alien.

If you're not checking APOD daily, why not?