It's not a new approach but actually a very old one. The author mentions Intellij but fails to note the history of Eclipse which, via IBM's Visual Age, you can actually trace back all the way to Smalltalk, which arguably had one of the first modern IDEs three decades ago.
Smalltalk programs were stored in a database. There were no .sm source files on disk anywhere. It had a refactoring browser, which was a tool that allowed you to restructure the code. This came straight out of the first research papers on refactoring. It came with a class hierarchy browser as well. It was pretty amazing for the time. Smalltalk was very clever about exposing its internals to developers. Refactoring, introspection, reflection etc. were all things that it supported out of the box. And meta programming, which is something that came out of the Lisp community, was also supported of course.
Visual Age was basically built around IBM's tooling for Java, which included an incremental compiler they built. A lot of the people involved with that had a Smalltalk background (it actually also was a Smalltalk IDE). It stored code in a database. Later, they created Eclipse which dropped the database but kept a lot of the compiler internals that allowed it to be way faster at refactoring and working with Java code than Intellij ever was. I use that every day and I still miss the two orders of magnitude difference in speed that Eclipse used to have. You must think I'm exaggerating. I'm not. Eclipse would be able to compile and be ready to run your code in between key presses. We're talking milliseconds here. In Intellij it's never less than 4-5 seconds and usually a lot more. Even on an M1. I have one. It helps. Faster is better. But it's still slow. It's just architected wrong to be that fast. It would need to internalize the compiler for it to be that fast and it just never did that. It relies on external build tools running via forked processes. It mitigates with a lot of (flaky) caching. That's why it has a top level menu option labeled "Invalidate Caches". Because cache coherence is a hard problem and they have plenty of bugs related to that.
So, it's an old idea and a very good idea. The ultimate version of this idea would be intentional programming (https://en.wikipedia.org/wiki/Intentional_programming) where the core idea is that programming is manipulating abstract syntax trees and that text is a mere serialization of that syntax tree. With intentional programming, you use tools (including editors) that do things with that syntax tree. Sadly, Simonyi, the person who came up with this never really got any traction with the company he founded out of Microsoft for this.
However, modern compiler design is finally starting to acknowledge that there is value in being IDE friendly. A compiler necessarily has to do a lot of the same things an IDE does but with very different requirements. Simply running a compiler from and IDE is slow and leads to a lot of repeated work. A much better approach is a compiler that exposes its internals to the IDE directly and runs incrementally. Like IBM did with Java in the late nineties. The Rust community has also been working on this lately and it's a topic that you see discussed in the context of other compilers. And with VS Code gaining popularity, a lot of languages are now supported through language servers that expose (some) basic refactoring functionality and other things. Mostly this is limited by the underlying tools; i.e. compilers and interpreters. It's nowhere near what Intellij does but it's better than nothing. And it's creating some stimulus for compiler makers to do better.
Jetbrains took their sweet time figuring that out (given they make IDEs and a major language) but their upcoming version of the Kotlin compiler frontend takes some steps in that direction as well. Java is slow in intellij. Kotlin is slower. Because the compiler is doing a lot of work and it can easily take something like half a minute to compile our code base; which isn't that large. I love Kotlin but I don't like how much time it sucks out of my day waiting for stuff to happen in intellij. And it needs to re-compile things quite often. The flipside is that it's a very good IDE that provides a lot of time saving functionality. I have a love / hate relationship with it. Love what it does, hate how slow it is doing it.
If you're comparing IntelliJ with any other major IDE, that's so incredibly wrong. Very bold of you to claim Eclipse is faster at anything. Everyone was using Eclipse 10 years ago, but it was just such a horrible experience that today the large majority of people are either on IntelliJ or even VSCode! WE didn't move because we liked to learn a new tool with different ways of doing things (IntelliJ used to be even more opinionated back then)... most of the devs I know only moved to IntelliJ reluctantly, forced to by everyone else who had already moved saying how much better, faster, polished it was... and once you moved, you would agree and convert more people because it was, and is, just that good.
I moved to intellij because Eclipse's gradle support was lacking and people seemed to no longer care about fixing it to e.g. run bog standard Spring Boot projects. So yes I did move, and no I don't agree with you on this. And after years of using it, it hasn't gotten much better.
Intellij is and was a resource hog. And slow. It's a recurring topic in the Intellij release notes. And it did get better over the years. There are numerous open/unresolved issues in their issue tracker related to all sorts of performance issues. No need for me to file new ones: they know. That's why they keep trying to fix and improve it. But the proper fix is going to require doing what Eclipse did two decades ago, which would be a major architecture change. I don't think they'll ever do that for Java. They might do that for Kotlin though as that is a compiler they've built. Not doing this from day 1 with Kotlin was a design mistake IMHO. But they are working on a more IDE friendly Kotlin compiler.
I've actually demonstrated how slow intellij is to people who told me I was crazy. On their own supposedly heavily optimized setups with their own projects. Edit some code, run some test. Eclipse: no delay whatsoever, test start running immediately. Intellij: it takes at least a few seconds before anything happens. That's the best case scenario. Change even one letter in the code and now it "incrementally" compiles, which in Intellij means firing up a gradle process. In Eclipse, the process already runs. It picks up the change immediately and then the single file that you changed is recompiled. It could even do this on broken code. So you could have compile issues in the file and still run a unit test against the non broken parts. Partial and incremental compilation are hard problems to solve. Eclipse had this, Intellij does not. It does some incremental stuff via gradle. But that takes a lot more time. About two orders of magnitude more. Even on a new M1 macbook pro.
I still don't understand why Eclipse as an open source project couldn't adapt. Unlike emacs, which is written in elisp, Eclipse is written in Java, the language in which its users are fluent. Why do developers move instead of adapting their IDE?
That doesn't explain why the community didn't step up much much more and brought the development to new heights.
What is holding back Java developers from using their Java language to adapt the IDE written in Java that they daily use? If their boss told them to implement a feature, they would do that. But they don't implement the feature when they need it.
It's a relative doddle to make changes or write an extension to Emacs or even a Smalltalk system like Pharo.
Eclipse, by comparison, is complicated. There's a whole lot of scaffolding and frameworkshit that goes into building an IDE that works the way Eclipse does, and that greatly increases the friction of extending it. Plus Eclipse, like most Java programs, is not really a live system that can be extended and changed as you run it; both Emacs and Smalltalk are. Not being live further increases friction. End users aren't really supposed to extend Eclipse themselves, just download and install prebuilt extensions, and this is accounted for in its design.
Visual Studio Code has the same problem. It may be somewhat easier to extend than Eclipse, and it was certainly intended to be so, but doing everything in webshit didn't really address the fundamental issue.
Hm, I actually liked eclipse, once I got a bit used to it. At least for a Java project. Mixed feelings about stability sometimes. Mixed feelings about intelliJ's bloat as well.
I can't speak to historical Smalltalk -- maybe Alan Kay or Dan Ingalls will come on thread to clarify -- but its lineal descendants, Squeak and Pharo, absolutely do store sources in separate files on disk -- for Squeak, in the SqueakVxx.sources and <image name>.changes files respectively. These aren't really databases, either, just blobs of Smalltalk source each of which contains class definitions and method updates linearly serialized; the Smalltalk system knows how to walk and index them to find the source for a particular method. Contrary to popular belief, the image is not used to store smalltalk source; if you remove either of these files, the Smalltalk system can recover source code for compiled methods in the image from the compiled bytecode, but things like local variable names will be lost. The source files are not intended to be hand-edited directly, of course, but you can, theoretically, do it.
In traditional Smalltalks, all the method objects have a shared object representing the file that method source is stored in (either .sources or .changes, with room for more) and an offset in the method to point to the text. On compilation, all it does is write the new method with metadata to the end of the .changes file, then set the text indexes. There is a mechanism to crunch out all the unused source text and generate a new merged .sources file. If you are all working on the same base version, .sources files can be shared since they are usually read only.
The nice part of doing this is you don't have to use much core for the method source, which was a big deal in 1978. You can easily strip the sources and changes files if you don't need them. eg. when distributing a build to customers. It also lets you remember the change history and enables recovery from crashes and other errors. The basic mechanism is also fairly easy to repurpose to support other change tracking approaches.
The interesting trick, is if you don't have a .sources or .changes, the system will run the Decompiler, which almost always produces readable code that is structured like the original source but with no comments and the local variable names changed to t0, t1, t2,... etc. There have been proposals to keep local var names in the image as to make it a bit less painful. It is surprisingly easy to code like this. There have also been proposals to store the whole source text in the image. You could do it and it wouldn't be hard, but you'd lose the advantages of the external code storage.
This does not count the various project facilities and change lists so you can keep related changes grouped.
I basically never saw any source files. According to wikipedia, you could export them though. But it's a long time ago and I never did much with smalltalk. I was still in university and this was part of our object oriented programming course.
A file that is easy to read and index sounds like a database to me. Not an sql database obviously.
VisualWorks was probably way different. But I've heard "Smalltalk had no source files, it stored all its program source in the image" enough times in a variety of contexts that I felt the need to address it.
`jillesvangurp` may be talking about ENVY/Developer -- "All code is stored in a central database rather than in files associated with a particular image." page 4
At least in my experience Eclipse was dog slow even on machines with sufficient resources. I switched from Eclipse to Intellij back then speicifcally coz Intellij was faster
The dumb editor parts are slower, but the smart refactoring parts are much faster. Eclipse understands your code while IntelliJ just manages it by passing it around.
Smalltalk programs were stored in a database. There were no .sm source files on disk anywhere. It had a refactoring browser, which was a tool that allowed you to restructure the code. This came straight out of the first research papers on refactoring. It came with a class hierarchy browser as well. It was pretty amazing for the time. Smalltalk was very clever about exposing its internals to developers. Refactoring, introspection, reflection etc. were all things that it supported out of the box. And meta programming, which is something that came out of the Lisp community, was also supported of course.
Visual Age was basically built around IBM's tooling for Java, which included an incremental compiler they built. A lot of the people involved with that had a Smalltalk background (it actually also was a Smalltalk IDE). It stored code in a database. Later, they created Eclipse which dropped the database but kept a lot of the compiler internals that allowed it to be way faster at refactoring and working with Java code than Intellij ever was. I use that every day and I still miss the two orders of magnitude difference in speed that Eclipse used to have. You must think I'm exaggerating. I'm not. Eclipse would be able to compile and be ready to run your code in between key presses. We're talking milliseconds here. In Intellij it's never less than 4-5 seconds and usually a lot more. Even on an M1. I have one. It helps. Faster is better. But it's still slow. It's just architected wrong to be that fast. It would need to internalize the compiler for it to be that fast and it just never did that. It relies on external build tools running via forked processes. It mitigates with a lot of (flaky) caching. That's why it has a top level menu option labeled "Invalidate Caches". Because cache coherence is a hard problem and they have plenty of bugs related to that.
So, it's an old idea and a very good idea. The ultimate version of this idea would be intentional programming (https://en.wikipedia.org/wiki/Intentional_programming) where the core idea is that programming is manipulating abstract syntax trees and that text is a mere serialization of that syntax tree. With intentional programming, you use tools (including editors) that do things with that syntax tree. Sadly, Simonyi, the person who came up with this never really got any traction with the company he founded out of Microsoft for this.
However, modern compiler design is finally starting to acknowledge that there is value in being IDE friendly. A compiler necessarily has to do a lot of the same things an IDE does but with very different requirements. Simply running a compiler from and IDE is slow and leads to a lot of repeated work. A much better approach is a compiler that exposes its internals to the IDE directly and runs incrementally. Like IBM did with Java in the late nineties. The Rust community has also been working on this lately and it's a topic that you see discussed in the context of other compilers. And with VS Code gaining popularity, a lot of languages are now supported through language servers that expose (some) basic refactoring functionality and other things. Mostly this is limited by the underlying tools; i.e. compilers and interpreters. It's nowhere near what Intellij does but it's better than nothing. And it's creating some stimulus for compiler makers to do better.
Jetbrains took their sweet time figuring that out (given they make IDEs and a major language) but their upcoming version of the Kotlin compiler frontend takes some steps in that direction as well. Java is slow in intellij. Kotlin is slower. Because the compiler is doing a lot of work and it can easily take something like half a minute to compile our code base; which isn't that large. I love Kotlin but I don't like how much time it sucks out of my day waiting for stuff to happen in intellij. And it needs to re-compile things quite often. The flipside is that it's a very good IDE that provides a lot of time saving functionality. I have a love / hate relationship with it. Love what it does, hate how slow it is doing it.