Extend TMS WEB Core with JS Libraries with Andrew: CodeMirror

TMS Software Delphi  Components
Do you want to display or edit code directly within your TMS WEB Core application? Then this is the post for you.  This time out, we’re diving head-first into CodeMirror 5, which describes itself as “a versatile text editor implemented in JavaScript for the browser.”  Which it certainly is.  Beyond just being a text editor, it is natively aware of more than 100 different programming languages.  Beyond that, it has numerous add-ons and configurable options to mimic the many styles and conventions of other editors you might already be familiar with, like Vim, Emacs, and so on. And using it in a TMS WEB Core project is just as easy as any of the other JS libraries we’ve covered so far.

Motivation.

Why do we need a code editor at all? The very first benefit of a code editor, as compared to a simple text editor or an editable TWebMemo field, for example, is that it typically comes with syntax-highlighting that is configured for the programming language that you’re editing. For Pascal, this means that begin and end are automatically shown in a different color, as well as any other Pascal-related reserved keywords.  This also typically means that the various kinds of brackets are shown in a way that makes it easy to find the matching pairs of brackets, or that comments show up in a different color or style.  Might not seem like much, but if you’ve ever tried writing code without this, it is a very different experience. Of course, you’re likely very familiar with this kind of thing, as the Delphi IDE does exactly this (and much, much more!) when editing code.  

Sometimes it is even helpful to display other text using this kind of mechanism, if for no reason other than to make it a little nicer to look at.  If you have log files that are potentially visible to your users, for example, they can be setup so that they are viewed through CodeMirror.  Perhaps set to something like SQL (my personal preference) will make the log files a little easier to read, depending on how they’ve been formatted, with dates and numbers and other things displayed potentially in a different color.  Or perhaps you want to display text that has line numbers beside it.  Or perhaps the most common of all, you want to edit HTML source and have the tags shown in a different color.  We’ll be taking a look at both Summernote and SunEditor in an upcoming post, both of which use CodeMirror to provide their “edit HTML source” functionality.

CodeMirror 5 vs. CodeMirror 6.

Before we get any further, let’s quickly sort this out.  Normally I’m 100% all-in when it comes to using the leading (bleeding) edge of any particular JS Library, with the thought that it will likely have the best support for the latest browsers and the most active development efforts, as compared to older versions of the same library.  And this does indeed apply here as well.  However, they’re a little too far out on the leading edge at the moment.  So far out that CodeMirror 6 is really a collection of modules that cannot be directly (perhaps easily would be more accurate) loaded into a web application, as we’ve been doing.  Instead, the various modules and their dependencies (and those modules’ dependencies, etc.) have to be combined into a package (using something like rollup or webpack) that can then be served up and used as a library by a typical web application.  And while that is a useful and interesting exercise all on its own, we’ll have to save that for a future blog post if there’s sufficient interest in it.

Getting Started.

Adding CodeMirror 5 to your TMS WEB Core project (the manual approach, as I’ve been referring to it) involves the same steps as any of the other JS Libraries we’ve covered – adding a JS and CSS link to your Project.html file, or using the Manage JavaScript Libraries feature within the IDE to do the same sort of thing. We’ll talk about the component approach that we used in FlatPickr (Part 2) at the end of this post. There is one difference, however, and it relates to the language(s) you want to include.  As so many languages are supported, the library would be too large if they were always all loaded, and particularly wasteful as there’s little need for most of them, most of the time.  Unless your project just happens to be focused on supporting a broad array of programming languages, naturally.  So in addition to the usual JS and CSS links, you’ll need to include links for the languages of interested to you.  So far, in my TMS WEB Core travels, the languages that have come up most often include Pascal, JSON, Markdown, SQL, CSS, HTML, and of course JavaScript. Let’s consider those, but by all means have a look at the complete list to see if there are others that are of interest to you.  

Note here that we’ve opted to specify @5 as the version, so we’ll get all the updates and patches to the CodeMirror 5 branch without having to worry about it suddenly being switched to CodeMirror 6, should they decide to release a particular configuration in this fashion in future.

Then, to see it in action, we need to link the CodeMirror library to an HTML element.  We’re not going to be too fancy here, but let’s put in a bit of effort, just so that we have some nice screenshots to include. We’re going to use two TWebHTMLDiv components, one called divBorder to give our editor some nice rounded corners, and within that a second TWebHTMLDiv which will be the mounting point for the CodeMirror editor itself.  We’ll also use a Bootstrap template to make this a little easier.  

For the divBorder component, we’ll set an ElementID of divBorder as well, and set its ElementClassName to be ’rounded border border-dark’. For divEditor, we’ll leave the ElementClassName blank but set the ElementID to divEditor.  The align property of divEditor has also been set to alClient so that it fills up the space of divBorder.  Then in our WebFormCreate, we’ll add the following code.

procedure TForm1.WebFormCreate(Sender: TObject);
begin

  asm
    this.editor = CodeMirror(document.getElementById('divEditor'), {
      lineNumbers: true
    });
  end;

  divEditor.ElementHandle.firstElementChild.classList.Add('w-100');
  divEditor.ElementHandle.firstElementChild.classList.Add('h-100');
  divEditor.ElementHandle.firstElementChild.classList.Add('rounded');

end;

Nothing too special about that, we’re just turning on line numbers as a check to be sure that we’re in a CodeMirror instance.  The extra classes being added afterwards are being added to the parent CodeMirror element that is created as a child of divEditor.  And all we’re doing there is ensuring that it takes up the space that divEditor has available to it (which in turn is the same size as divBorder) with the same rounding, so we have nice clean-looking corners.  If we then add in some sample text (with line breaks added manually here), we get the following.

 

TMS Software Delphi  Components

The scrollbar on the right appears when needed, and the horizontal scrollbar appears as well, if the lines stretch beyond where they are now.  Note that this is a screenshot from Chrome (v102).  Looking at the same control in FireFox (v100), it is pretty much the same thing but with a different style to the scrollbar.  We’ll get to that a bit later.

 

TMS Software Delphi  Components

Also note that some words, like ‘do’ and ‘in’ have a different color.  This is CodeMirror already trying to apply syntax highlighting, which will be Pascal, as that was the first ‘mode’ that we listed in our list of supported languages. As ‘do’ and ‘in’ are keywords in Pascal, they are being highlighted here.

Coding Language = Mode.

In CodeMirror, the ‘mode’ refers to a particular coding language JS file that is being used to apply things like syntax-highlighting.  It can be specified explicitly when the CodeMirror instance is created, by passing a mode: option.  To get a feel for what this all looks like, here are some examples where the mode was explicitly set, and a different sample was added to the editor.  

TMS Software Delphi  Components

mode: “pascal”

TMS Software Delphi  Components

mode: {name: “javascript”, json: true}

TMS Software Delphi  Components  

mode: “markdown”

TMS Software Delphi  Components  

mode: “sql”

TMS Software Delphi  Components  

mode: “css”

TMS Software Delphi  Components

 

mode: {name:”xml”, htmlMode:true}

TMS Software Delphi  Components  

mode: “javascript”

Note that these different modes alter slightly how text is entered. For example, changing the syntax-highlighting on the fly when matching pairs of elements are found.  Or adding the proper indent when closing a matching element. Markdown is probably the most dramatic in how it handles all of this.

Split Personalities.

Typically, a section of code will represent a single programming language, and a single mode will be sufficient for editing purposes.  But there are instances where multiple languages are used within a single file, and thus multiple modes would be useful to have.  The most common situation is when HTML, CSS, and JavaScript are intermingled within the same file.  To handle this scenario, CodeMirror has another mode that can be included. It relies on the XML, JavaScript and CSS modes being available as well.

TMS Software Delphi  Components  

mode: “htmlmixed”

Does another example jump immediately to mind? One source file, multiple programming languages?  Yep, one sure does.  Pascal with embedded JavaScript!  The Delphi IDE, as fantastic as it is, isn’t particularly adept at handling exactly this situation.  CodeMirror naturally hasn’t any clue that someone would want such a combination either, but it does have a more generic “overlay” mechanism that essentially allows us to do the same thing. To make use of it, we need to include an “Addon” script in our Project.html and then we have to tell it what we want it to do.  By default we want it to be in “pascal” mode and we want to switch to “javascript” mode when we encounter an “asm” token (open), and then stop “javascript” mode and return to “pascal” mode when we encounter an “end;” token (close).  First, the extra file we need is this one.

  

Now we just need a bit of code to tell it how to switch modes to “javascript” and when to switch back.

  asm

    CodeMirror.defineMode("asm", function(config) {
      return CodeMirror.multiplexingMode(
        CodeMirror.getMode(config, "text/x-pascal"),
        { open: "asm",
          close: "end;",
          mode: CodeMirror.getMode(config, "text/javascript"),
          delimStyle: "delimit"
        }
      );
    });

    this.editor = CodeMirror(document.getElementById('divEditor'), {
      mode: "asm",
      lineNumbers: true
    });
  end;

The end result is that CodeMirror switches back and forth as needed.  Other languages could also be added in the same way, just by adding more stanzas to the definition.

TMS Software Delphi  Components  

mode: “asm”

Beyond this, there are other tweaks possible.  For example, updating the Pascal mode to include “asm” as a reserved keyword or changing how the delimiters are handled (whether they are handled as Pascal or JavaScript, for example). It might also be helpful to use a different delimiter.  Perhaps using asm blocks defined with  asm { and } end; or even with {$IFNDEF WIN32) asm { and } end {$ENDIF} as ways of providing a more solid break between Pascal and JavaScript code. This may be particularly helpful when JavaScript has numerous (potentially unmatched) brackets and maybe even an “end;” appearing in its block that belongs there, but doesn’t necessarily mean the end of the asm block.  It happens, honest!  Of course, ultimately the Delphi IDE still needs to be able to compile the resulting code, so you can’t go too crazy here.

So. Many. Themes.

I’m not sure why there are so many themes for CodeMirror, but there are more than 50.  Ace Editor has nearly 40.  So many themes!  And while it might seem like you’re sure to find one to your linking, this is probably not going to be the case if you’re trying to match CodeMirror to your own theme.  They aren’t wildly different from one another – mostly light and dark and a few variations of the colors within.  Easy enough to use them though.  Two steps.  Find one you like, and add a corresponding CSS link to your Project.html file. 

 

 

And then specify the theme you’d like as one of the options.

 

 

  this.editor = CodeMirror(document.getElementById('divEditor'), {
      mode: "asm",
      theme: "darcula",
      lineNumbers: true
    });
 

This means that you can actually load up multiple themes, if you’re so inclined, and then apply different themes to different CodeMirror instances. For example, maybe you want something dark when editing SQL and something light when editing Pascal.  Options abound!  Here’s what “darcula” looks like.

TMS Software Delphi  Components  

theme: “darcula”

Beyond themes, CodeMirror generates 100% HTML and CSS code, without any pesky tags, so you’re free to customize it further all you like, using CSS to override very nearly everything.  All of the themes are, themselves, just CSS files and not terribly complex ones at that.  So the easiest approach is likely to find a theme that is somewhat close to what you’d like, and then tweak that CSS file, either embedding it in your own CSS file, or by creating a new theme and adding it to your project using the same steps as above for the existing themes.

VI vs. EMACS 2022.

There is of course much more to CodeMirror than a little syntax highlighting, line numbers and some color here and there.  Some things it can do are a little bit bonkers.  And some are exactly what you’d expect from any modern text editor.  

First, the bonkers.  If you’re old like me, you’ve used vi as an editor since around about the time you learned what an editor was.  Or maybe you’re one of those people who grew up on the emacs side of that particular holy war (“a great operating system, lacking only a decent editor”).  No matter which side you defended, decades ago, both are still here today.  And so are other editors that can be emulated in the same way, like Sublime.  But we’re only going to cover vi because well, it’s pretty much the same effort to configure either one anyway. Actually using them is a different story!  As with themes, and as with much of what we’ve got left to cover, there’s a two-step approach.  Add the necessary JS file to your Project.html, and then set the option when the CodeMirror instance is created.  Naturally, I’d not be setting this particular option on any CodeMirror instance that is available to any unsuspecting users as that would likely cause no end of headaches.  Or humor, depending on the audience. But if you’re game, here’s all there is to it.

 

 

and then

   

this.editor = CodeMirror(document.getElementById('divEditor'), {
      mode: "asm",
      theme: "darcula",
      lineNumbers: true,
      keyMap: "vim"
    });

And you’re off and running.  A little bizarre, but functional.  CodeMirror doesn’t make any claims about this being complete, but it does have support for all the basic things, including search/replace, copy/paste and basic navigation commands.  Be sure to check out their demo of this feature and bear in mind you may have to add in some other bits to get the full effect.  For example, adding in a couple of more elements to show the current editing mode, and so on. Emacs support can be added the same way (just replace vim with emacs in the above), but do check out the notes on that.  Seems some of the most common Emacs key bindings are likely going to make you very unhappy, given their browser equivalents!  Which, if memory serves, were both huge ‘issues’ back in the day – vi inducing schizophrenia from its different ‘modes’ and Emacs being, well, Emacs, with its far-too-many modifier keys for everything imaginable.  As the saying goes, the more things change, the more they stay the same.  

Searching and Replacing.

Speaking of which, searching and replacing text is a pretty common activity when using any text editor. Here, the main trouble is that we need some kind of interface, aka a dialog, to get the information we need.  So a little more fiddling, but doable all the same.  Rather than go through these individually, let’s just treat them as a set.  And we don’t even need to modify the code that creates the instance – having these added does all the work for us.

 

 

With those in place, you’ve got a bunch more keyboard shortcuts at your disposal.  Starting with Ctrl+F to start a search, Shift+Ctrl+R for global search/replace, and Alt+G for “goto line number:column” support.  There’s more, so be sure to check out this page to see the rest, and to try them out in a live demo environment. 

Scrolling, Scrolling, Scrolling.

About those scrollbars.  Browser compatibility has grown in leaps and bounds over the years, and we’re almost at the point that, most of the time, you can look at a web page and not even know which browser rendered it.  This is helped by there being so few engines left.  But even still, the “browser chrome” – referring to the native OS parts of the browser still visible, not the Google Chrome browser, still tends to give away which browser you’re using.  And even if you hide the title bar, you’re still left with one glaring difference – the scrollbars.  These are still rendered differently (or maybe not even rendered at all) depending on the particular combination of browser vendor and operating system.  The folks working on CodeMirror had clearly had about enough of this, and have added their own scrollbar overrides.  Which sounds great (and actually is great!) but of course you’re likely stuck with other non-CodeMirror things that have scrollbars, so while this might be great for your CodeMirror instances, it isn’t going to help if you have several things on the same page with different styles of scrollbars.  But never mind that!  Let’s give one a try.  The “simplescrollbar” code is what we’re after, and we’ll throw in the “scrollpastend” for the same low, low price here:

 

 

Then, in the CodeMirror initialization code, we need to add something to tell it about the new scrollbars. And then voila! We get a nicer looking scrollbar that (hopefully) works and looks the same, across browsers and operating systems. Be sure to test though!  If you wanted to change the color of the scrollbars, it is just a bit more CSS. Or if you don’t want to use CSS, you can set it programmatically like this.

 

   

 this.editor = CodeMirror(document.getElementById('divEditor'), {
      mode: "asm",
      lineNumbers: true,
      scrollbarStyle: "overlay"
    });

    var scrollbars = document.querySelectorAll(".CodeMirror-overlayscroll-horizontal,.CodeMirror-overlayscroll-vertical");
    scrollbars.forEach(function (el) {
      el.firstElementChild.style.backgroundColor = 'darkorange';
    });

 

The result should then look something like this. 

TMS Software Delphi  Components  

There are lots of other CSS tweaks that you can add as well, impacting the size and so on.  Here’s what I’ve got showing on a login page, where the color matches the overall theme for the site (that’s suppose to be the color of tree bark…) with a wider bar and larger offset from the edge. Which is a bit of a sleight-of-hand as the border is just further away from the control itself which has no border.  

TMS Software Delphi  Components  

Logging, Logging, Logging.

Another solid use-case for something like CodeMirror is similar to the above – a place in which to log various bits of data as they happen. In such cases, it is likely that you’ll want to do two things.  First, append a new entry to whatever text is currently displayed. And second, scroll the content so that you can see the latest entry at the bottom, as it happens.  Kind of like “tail -f” for those who are so inclined.  Here’s an example where a timer is used to append text.

 

procedure TForm2.WebTimer1Timer(Sender: TObject);
var
  addthis: String;
begin
  addthis := chr(10)+FormatDateTime('hh:nn:ss.zzz',Now)+' Say something interesting';
  asm
    this.editor.replaceRange(addthis, {line: Infinity});
    this.editor.execCommand("goDocEnd");
  end;
end;

 

If you already have text you want to load into CodeMirror, you can use something like this.editor.setValue(text), where text can include types like WideString. 

More of Everything.

With the basics out of the way, there is still a ton of other features to be found.  There are commands that you can use to highlight text, or even mark parts of it as read-only.  Code folding is a big topic, with plenty of examples.  Same goes for code completion.  And dealing with brackets (matching or automatically adding them) is another one.  For the most part, this involves the same steps we’ve already covered.  Adding in a JS library or a CSS file to Project.html and then setting up some options in the CodeMirror initialization.  

But there’s a bunch of things you can do that are just commands, like we’ve done above with loading content, adding content, or shifting the viewport around.  Rather than just mere properties at initialization, these are things you can do to actively interact with both the contents of CodeMirror as well as with whomever is using it.  Clipboard actions fall in this category, as well as dealing with either getting selected text or actively selecting text automatically. Or inserting text at a particular position.  Lots of options here, depending on what it is you’re trying to do.

What CodeMirror Doesn’t Do.

Just to save you some trouble searching, CodeMirror is by no means perfect.  There are a few little things that I really, really wish it could do, but that it doesn’t.  Given that the project is open source, these are things that could eventually be created as an Addon of some kind or tweaked for a particular installation.  But I’m mentioning them here only to save you the trouble of looking for them.  Doubtful they’ll ever make an appearance in CodeMirror 5 with CodeMirror 6 motoring along as it is.

  • Virtual Spaces.  If you know what this is, you’d likely never settle for an editor without it.  And I’d agree with you 100%.  This refers to how, when moving the cursor up and down through a block of text, the cursor stays in the same column regardless of the length of text in each row.  So if your cursor is at the end of a line of text that has 50 characters, and then you cursor-up to a line that has only 2 characters, the cursor will remain in column 50.  This is how the Delphi IDE works. Notepad++ does this (or at least it has an option to enable it – been so long I can’t recall if it is enabled by default) and any self-respecting IDE editor should also have this.  But CodeMirror does not. Minus 10 points!
  • Block lines.  While CodeMirror has a lot of code-folding features, it doesn’t (as near as I can tell) have the ability to draw lines connecting block delimiters.  For example, in the Delphi IDE, if you have a begin/end block, you’ll see a line connecting the two.  So if you format your code as any sane person should, nested blocks will have a series of parallel vertical lines showing that nesting, making it easy to see when you’re missing an ‘end’ or perhaps you have too many. 
  • Code completion.  There are some code-completion features available through a few Addons.  And while these work, they aren’t really in the same ballpark of what we’re accustomed to in the Delphi IDE (when code-completion is working properly, that is!).  Using CodeMirror as a complete replacement for the Delphi IDE code editor is likely a tall order mostly because of this, if that was something someone was maybe perhaps potentially thinking a little bit about doing.  You could get close, but it would take a bit of work to make it useful in this fashion.

But, despite these shortcomings, I’m doubtful there’s another JS Library out there that is as capable as CodeMirror, or as popular in terms of being integrated with other JS libraries you might already be using in your projects.

Hey! What about Ace Editor? Or Monaco?

Yep, there are other JS Libraries that you can use to fill this particular need, certainly.  Ace Editor is even included as a component within TMS WEB Core.  And it is a really good library, as well.  I used it initially in some of my first TMS WEB Core projects, and managed to get it to do all the things I wanted it to do.  But ultimately it was swapped out in favor of CodeMirror, for two reasons.  One silly and one not so silly.

The silly reason was that, whenever I loaded up a block of text into Ace Editor, any text that was wider than a certain portion of the width of the component on the page would result in a horizontal scrollbar.  And I loathe horizontal scrollbars entirely!  And I don’t mean a scrollbar would appear at 98% or 99%.  It would appear at 60% or so.  Super annoying!  Sure, I could get rid of the horizontal scrollbar altogether, but I could never figure out how to get it to only appear when the line of text was actually longer than the width of the component. Something to do with a combination of the fonts I was using and who knows what else.  Some time has passed since then, and my CSS knowledge is a little different now than it was then, so I’m calling this silly as it is probably solvable.  But I put in the hours then and couldn’t seem to figure out why it was misbehaving in this fashion.

The not so silly reason is that I use other JS libraries that, themselves, use CodeMirror to provide additional functionality.  As mentioned at the outset, the two HTML editors that I’ve used most in TMS WEB Core projects are Summernote and SunEditor.  Both of these rely on CodeMirror to provide their “view as HTML” functionality.  It would seem less than ideal to have another library to perform the same task if CodeMirror is already required.  Other HTML editors, like CKEditor and TinyMCE, for example, also have CodeMirror plugins, so this isn’t an obscure practice by any means.

I’ve not used Monaco, but it did come up in some searches. If you’re on the hunt for a JS library for this kind of thing, I’d suggest CodeMirror as a solid contender, but by all means do have a look around and see what is available.  In head-to-head comparisons of late, CodeMirror 6 seems to be the leading candidate, so it might be time to figure out what all this module stuff is about and give that one a shot as well.

JSExtend Component ? 

I’ve not yet added CodeMirror as a component to JSExtend, but it is on the to-do list.  It may also be a good opportunity to build it as a CodeMirror 6 component.  All the rollup/webpack stuff can be done separately and then added directly as a package to that repository, making things a little easier for everyone. Potentially. Let me know what you think would be of interest. Next time out, we’ll actually be looking at Summernote and SunEditor, so we’ll get a bit more involved with CodeMirror integration there as well.  

Andrew Simard.