The ability to create a modular system does have significant benefits, especially in business. The ability to remove and replace individual components keeps costs low while allowing incremental improvements in both speed and efficiency. However, as with most things, there is no free lunch. The modularity provided by the Von Neumann architecture comes with some serious deficiencies:
- Von Neumann bottleneck: Of all the deficiencies, the Von Neumann bottleneck is the most serious when considering the requirements of disciplines such as AI, machine learning, and even data science.
- Single points of failure: Any loss of connectivity with the bus necessarily means that the computer fails immediately, rather than gracefully. Even in systems with multiple processors, the loss of a single process, which should simply produce a loss of capability, instead inflicts complete system failure. The same problem occurs with the loss of other system components: Instead of reducing functionality, the entire system fails. Given that AI often requires continuous system operation, the potential for serious consequences escalates with the manner in which an application relies on the hardware.
- Single-mindedness: The Von Neumann bus can either retrieve an instruction or retrieve the data required to execute the instruction, but it can’t do both. Consequently, when data retrieval requires several bus cycles, the processor remains idle, reducing its ability to perform instruction-intensive AI tasks even more.
- Tasking: When the brain performs a task, a number of synapses fire at one time, allowing simultaneous execution of multiple operations. The original Von Neumann design allowed just one operation at a time, and only after the system retrieved both the required instruction and data. Computers today typically have multiple cores, which allow simultaneous execution of operations in each core. However, application code must specifically address this requirement, so the functionality often remains unused.
