The article starts out with a simple, incremental approach, pretty easy to grasp the concepts all the way through the discussion of the lexer and digesting the program input.
Then all of a sudden the density of the material accelerates dramatically. The discussion of the parser, and parsing as it applies to the language, becomes very opaque to readers not already familiar with the technologies being described.
It's perfectly OK to proceed that way for a sophisticated audience, but the first part of the article is written at a much more introductory level. An unsuspecting beginner may very likely feel overwhelmed about half-way through, and probably give up at that point.
When I got to the explanation of the AST, no surprise, it was exactly representable as an sexpr, which anyone with a few hours experience with Lisp or Scheme would recognize. Not sure, maybe it would have made more sense to begin with the parsed goal (the AST) and work backwards to explicate how parsing of the original syntax was done.
I guess parsing is a really hard subject to teach, but it's the core idea that the reader needs to grasp to be able to understand PL construction as the author laid it out. A still gentler introduction could be possible, if it doesn't sound so easy to do.
I noticed this phenomena in my own writing and I think I know where it comes from. In Polya's list of heuristic (from How to Solve It) he writes about the the two rules of teaching:
> The first rule of teaching is to know what you are supposed to teach. The second rule of teaching is to know a little more than what you are supposed to teach.
When we learn something and then write about it to teach others, oftentimes we are excited about the things we just learned, so we write about that, even if we don't fully understand it yet. Then we add an introduction to the "meat" (i.e. the limits of our current understanding) which is very clear, precisely because we are "overqualified" to teach it. Another way to state Poyla's rule for teaching is as follows:
Being overqualified in knowledge is a prerequisite for being qualified as a teacher of that knowledge.
One way I like to picture knowledge is as a dark room. You might step into one but that doesn't mean you know every corner of it. For example, imagine a proof or a technique. Unless you can know by heart why each assumption is there, or when the technique fails, you don't fully understand it. If you are guiding someone through a dark house, as you go from room to room, you are only qualified to give someone directions about the room you already been in, not the one you are in right now - even if that's what is on your mind.
Note that this isn't unique to people writing technical blogs, it happens in academia a lot too. There's a reason Feynman's lecture on "undergraduate physics" attracted the attention of graduate students.
Your comment resonates very weirdly with my experience.
College teachers are guilty of the exponential wall. Easy and lengthy intro then steep but short core material (often cut short because the bell is ringing). Impossibly annoying.
The dark room exhaustive search is exactly how I ended up learning about learning. Through music though, as a self taught I had to visit every possible spots, even if most of them were holes. And most of the time, most things only 'make sense' because the other things just don't work. Now if you teach a topic to someone by listing what 'make sense', unless that person is very easy on remembering or extremely creative, it will be a meaningless burden to rote learn. On the other hand, knowing all the wrong parts, makes you very good at teaching someone since as the student will ask why his intuition is failing, you know how it feels and you know what to do about it, how to try other ideas, patterns, revisit things that he might have overseen.
Douglas Crockford said that once you understand monads, you suffer the curse of not being able to explain it anymore. I really wonder how often this happen, how many students learned shortened and polished stuff in books without the whole story, the whole experience of the actual researcher ? This tiny offset in understanding, repeated generation after generation .. might lead to large divergence. Saying this after watching (too many?) A. Kay talks about how people ignore the past, they ignore it because it's not taught that is all.
I started writing this piece because a friend of mine recommended I write an article about Duck and post it on Reddit. I don't do a lot of writing and I started thinking about it and it seemed like it would really be a huge task.
HN crowd is a little bit more "sophisticated" so maybe that helps, but writing it I felt like it was going to be harder to get to the end when I started, so I really skipped over all of the details at the end. I'd like to give a better explanation of both the parsing step (to an understandable degree) and the interpreter itself, so that it is easier to follow along and implement the steps.
When I developed the project, it was sort of a stop-and-go process. I wrote an LR parser because I wanted to brush up on topics from a compilers course and it seemed like the most exhaustive way to do that. With adjustments, I can use the tables generated with any language to work on my next programming languages project.
After finishing the parser/parser generator, I was mainly throwing context-free grammars at the wall to see what was working or wasn't, without really analyzing them to see if the fit in the LR(1) category. I used a mini Java CFG I had from the compilers course and played around with that for a while, and I started working on a C grammar but that became annoyingly complex. Duck represented a kind of pseudocode that I could use.
Eventually I came back to it, maybe about a year or two later, and started to realize how easy it would be to get this thing able to run programs. So in about the course of a week I had the initial runtime functional.
I appreciate any advice for ways to explain it that might be better. I think this piece could use some editing. I would like to expand parts, I'm just concerned that it would be too long to read then. I think the initial suggestion was supposed to be a brief explanation, but I thought the only way to write this would be to go over everything completely.
I'll be sure to go back and improve this as it progresses. I've also been meaning to make improvements to the programming language itself, so I need to find a way to balance my spare time.
> ... so I really skipped over all of the details at the end. I'd like to give a better explanation of both the parsing step (to an understandable degree) and the interpreter itself, so that it is easier to follow along and implement the steps.
That pretty well describes the problem and the way to make it more digestible. Giving a step-by-step explanation of the parser will be very useful. Illustrating what the parser is doing with clear examples would be very helpful.
Reading it again, I think the "sticky" parts begin with discussion of the grammar. A couple of examples, then the reader encounters a blizzard of phrases, "<terminal>", "<non-terminal>", "<S>", etc., strewn through the explanatory paragraphs which rapidly become confusing. Readers may be wondering "what 'production' are we talking about now?" and so on. Once losing the thread it's impossible to follow the text all the way down.
True, HN attracts a more knowledgeable crowd. Even so I'm sure there are many here who are not that familiar with the intricacies of parsing but want to gain a better grasp of it. Condensing the article's preliminaries, while expanding the parsing (and subsequent) sections and slowing down the pace, would likely make it better for both author and the reader.
I've only skimmed the beginning of the article but it seems to me that this is not about designing a language but about implementing a language. If you are going to design a language you have to consider a lot more than static versus dynamic.
I would say that the two main camps were typed versus untyped not static versus dynamic.
What about functional versus imperative versus actor.
So, in my opinion it dives much too deep into implementation much to quickly without any meaningful discussion of the design of the language or the process of designing a language.
Then all of a sudden the density of the material accelerates dramatically. The discussion of the parser, and parsing as it applies to the language, becomes very opaque to readers not already familiar with the technologies being described.
It's perfectly OK to proceed that way for a sophisticated audience, but the first part of the article is written at a much more introductory level. An unsuspecting beginner may very likely feel overwhelmed about half-way through, and probably give up at that point.
When I got to the explanation of the AST, no surprise, it was exactly representable as an sexpr, which anyone with a few hours experience with Lisp or Scheme would recognize. Not sure, maybe it would have made more sense to begin with the parsed goal (the AST) and work backwards to explicate how parsing of the original syntax was done.
I guess parsing is a really hard subject to teach, but it's the core idea that the reader needs to grasp to be able to understand PL construction as the author laid it out. A still gentler introduction could be possible, if it doesn't sound so easy to do.