Takeaway: Analog computers are limited in precision and by "analog noise"; the precision of the components used determine the precision of the output. Usually no more than 3 or 4 decimal places are possible, at least with the tech that was used in their heydey. I would say that is still close to the case even today. Of course, one could do things like cryogenic cooling or such, but it becomes a cost factor at that point.
Something else that wasn't mentioned:
The ADALINE/MADALINE memister technology is an analog component, and is a hardware component equivalent to a McCulloch–Pitts neuron model (perceptron).
The memister is NOT to be confused with the memristor, which is a different technology; the memister was a 3-terminal device (memory transistor):
The high point of analogue computing for control systems may have been Concorde. It normally operated in fly-by-wire through an analogue interconnection: the so-called synchro/resolver system, which is a AC servo control system. The flight computers (mostly but not entirely analogue) provided autothrottle and autostabilisation.
Reminded me of A.K. Dewdney's "Computer Recreations" column in "Scientific American" back in the day (I think a lot of people may have hated on his column since it followed in the shadow of Martin Gardener's infamous "Mathematical Recreations"). One of the "analog computers" he mentioned was using dried spaghetti to sort numbers — where the length of each spaghetti noodle represented the magnitude of a number. Of course grabbing them into a bundle in your hand and setting the bundle on end on a flat surface would "sort" the values.
Would the surface tension across a film that accurately finds the shortest path between n points be considered a "computation"?
One thing fascinating by "analog computers" is the way they seem to be practically instantaneous regardless of n. That is perhaps part of the efficiency reflected in the article.
> One thing fascinating by "analog computers" is the way they seem to be practically instantaneous regardless of n. That is perhaps part of the efficiency reflected in the article.
If this were true, it would have profound implications. It's probably not true.
His soap bubble analogy is pretty cool. There is a direct analogy for filtering where its impossible to get instantaneous perfect response out of a filter, either DSP simulated or hardware. It would seem that merely filtering a signal is much simpler than simulating the airflow across a wing or whatever.
There are also conceptual issues that the airflow across a wing is ideally under stable low angles of attack, constant, but during "fun times" which is precisely when you'd want a model instead of a test pilot, the air flow will vary over time (to the general detriment of flying ability...) so what it means to instantly solve a wing is unclear in itself.
> Usually no more than 3 or 4 decimal places are possible
By that do you mean accurate to 1 part in 100 (3dp) or 1000 (4dp) or what? Since the scale of a representation is arbitrary, I'm not sure what dp means here.
Good question. I guess precision is finally going to be fractions of the max voltage swing allowed by the computer. For example, if voltage goes from -5 to +5 volts, the voltage swing is 10v, and if noise allows 0.1mV of precision, then the precision is 1/100000 of the full voltage swing.
This could be expressed, at the end, simply in decibels, though. Signal-to-noise, as in classic analog systems.
I think it usually refers to accuracy out of a range of 1. typically 3 decimal places means 1000ppm and 4dp means 100ppm.
The typical problems with analog computers are many...
precision of components (e.g. gain or attenuation) is limited to ~0.1% for resistors and ~1% for capacitors (inductors aren't typically used). You can try to tune things (ratiometrically) to get higher accuracy, but at the cost of increased noise and temperature sensitivity. The more complex the system, the more things can go wrong... so you end up needing simple systems or simple tools (digital).
The typical problem is that if you build a filter (e.g. a transfer function with a summer or differencer) then you will tend to clip the dynamic range or either with a maximum voltage (integrators) or a minimum noise level (differentiators) pretty quickly. You can play some games with log converters, but accuracy really still matters and drift or gain error with time is rarely an option.
The best way to use analog computers is with negative feedback to null the input. They do that amazingly well... so you can build a temperature controller, missile tracker, or actuator that only minimizes an error so that high gain corrects for any inaccuracy or offset.
A big technical EE problem for analog computers is interconnects and their EMI/EMC interference issues and impedance issues. The analog specs for on-chip digital circuitry are much more relaxing to develop around. You can work around the interconnect issues on analog computers by dumping lots of power into the driver and input circuits but eventually some joker is going to point out that it would be electrically cheaper (in terms of current/power draw, etc) to transmit that 0 to 5 volt signal using something like I2C or SPI and then you're on a fast slippery slope to turning your analog computer into an exercise in DSP programming. At some point of complexity the interconnect cable driver circuitry is going to be power hungry enough that its cheaper to emulate the whole thing in floating point on a digital computer.
If you make a graph of PITA vs bit resolution, we're all pretty comfortable emulating digital computers on analog real world circuits using binary ones and zeros. Surely the gain is very little and the PITA increases very much by implementing digital computers on trinary + - 0 analog computers. Some think the graph is U shaped and at some resolution level, the PITA of analog high resolution falls beneath performance so it makes sense. Many like me think that graph never U shapes such that anything is "better" at emulating digital computers than using analog physical computers based on binary 0/1. AFAIK no one has built a modern floating point accelerator using opamps and A/D and D/A converters, so I find it unlikely its useful.
A two transistor NAND gate is after all just a analog computer using simple binary signals. All computers are analog its just the popular digital ones are only defined and well behaved when using binary analog signals.
There is some audiophile effect going on. Surely a mp3 codec running on a vacuum tube opamp would sound more mellow and all that.
Now, I won't argue with cheaper to develop since analog is manual with a lot of issues to contend with. I'm just wondering if there are more applications that can get huge speedups at lower power or cost than digital. I know the ASIC makers in power-sensitive spaces are already moving parts of their chips to analog for power reduction. That's what mixed-signal people tell me anyway: the specifics are often secret. So, I have to dig into CompSci looking for what they've tried.
The analogue neural net stuff is quite a reasonable example, because it's specifically trying to mimic a real analogue system and tends to be noise-tolerant.
Couple of points from your lower link:
- return of "wafer-scale"! Nice.
- " the average power consumption is expected to stay below 1 kW for a single wafer"; not bad but you're still going to need to cool that
- actually a hybrid system: long range comms is digital and multiplexed to save wiring, converted to analogue at the synapse
- "All analog parameters are stored in non-volatile
single-poly floating-gate analog storage cells developed for
the
FACETS
project" => basically analogue Flash? A development of MLC I suppose
On reading the whole thing, it seems the magic is actually in choosing which bits to make digital. The "long range" neural events are sent as differential 6-bit bursts, multiplexed, which they claim saves significant power.
> AFAIK no one has built a modern floating point accelerator using opamps and A/D and D/A converters,
This is one of the smartest things i've read on HN. I guess you are correct. Although who knows, perhaps a differential equation solver could be faster using D/A -> analog computer -> A/D?
Don't forget the temperature compensation! Then there's irreducible noise like Johnson noise. As you say, the best use is in (properly stabilised) feedback systems which seek to minimise a difference.
Kind of seems like if you are measuring the analog computers virility by the number of decimal places it can represent maybe you are mis-using the machine. I mean how many decimal places can you or I do in our head in real-time?
I agree, but I don't think it is "calculated" in decimal places (if the makes sense). Sort of like how slide rules didn't give you "decimal precision".
Decimal places are just a convenient shorthand for describing the rough order of magnitude of the available precision. "Three decimal places" means roughly 30dB or 0.1%.
Slide rules absolutely give you decimal places. A decent slide rule might give you three decimal places of accuracy. A really good one might give you six decimal places, or 0.0001%, or 60dB. You could more precisely quantify their accuracy than just a rough order of magnitude, so perhaps the accuracy would be 55dB or 62dB, but "decimal places" gives you a sufficiently good idea of the accuracy for most purposes.
To bring it back to the digital comparison, a really great slide rule that's accurate to six decimal places is equivalent to a digital computer with 20 bits of output. If you put in a ton of work building an incredibly precise slide rule you might be able to add another order of magnitude and get seven decimal places. On the digital side, you'd only need to add 3 or 4 more bits to match that improvement.
I seem to recall there were clever but well-known techniques in analog to get higher accuracy than that of your actual components, through negative feedback IIRC. So why is it correct to say that the precision of the components is what limits output precision? Wouldn't the technique potentially make a difference? (and yeah I know accuracy != precision but I'm using them loosely... the distinction doesn't seem relevant here)
I haven't thought it through, but feedback lets you do a few (perhaps connected?) things: 1. explore a trade-off between gain and bandwidth, 2. Reject disturbances and nonlinearities.
So you could have a high gain but "low precision" (in the sense of deviating from an ideal, not in the sense of not being noisy) component, and through feedback you can make a low gain, high precision (having desired properties, not low noise) component.
I don't know the math of such things but did take a stab at it. My idea was doing something similar as we do for high or unlimited precision on digital computers. They usually emulate the higher precision using a series of lower-precision, primitive operations. My thought was that you could probably implement higher precision in analog if you could do a similar emulator with operations acting within the precision common in analog components. All I could guess at, though, since I'm in over my head here.
One other thing I always note is the brain seems to be mostly analog. Look what all it can do which includes memory and high-precision math. So, there's almost certainly some tricks we can use to do something similar with analog. Maybe an analog/digital hybrid. The wafer-scale project on NN's shows the potential esp if it was made 3D w/ a cooling system:
https://en.wikipedia.org/wiki/Analog_computer
Takeaway: Analog computers are limited in precision and by "analog noise"; the precision of the components used determine the precision of the output. Usually no more than 3 or 4 decimal places are possible, at least with the tech that was used in their heydey. I would say that is still close to the case even today. Of course, one could do things like cryogenic cooling or such, but it becomes a cost factor at that point.
Something else that wasn't mentioned:
The ADALINE/MADALINE memister technology is an analog component, and is a hardware component equivalent to a McCulloch–Pitts neuron model (perceptron).
The memister is NOT to be confused with the memristor, which is a different technology; the memister was a 3-terminal device (memory transistor):
https://en.wikipedia.org/wiki/ADALINE
...whereas a memristor is a two terminal device ("memory resistor"):
https://en.wikipedia.org/wiki/Memristor