Sciter v.2 SDK preview is available here:terrainformatica.com/sciter/sciter2-tech-preview.zip

Graphics

The whole Graphics subsystem was redesigned to use Direct2D backend. And so drawing principles were changed. Sciter v.1 uses <canvas> HTML5 alike model that implies bitmap buffer to be created for the element.

Such bitmap model contradicts CSS transforms – when you have something rendered on the <canvas> and use something like transform:rotate(15deg) the tranformation will be made on the whole bitmap rather on per primitive basis – line, rectangle, ellipse drawn.

Sciter2 uses so called immediate mode drawing thus if you define this:

someEl.paintContent = function(gfx)
{
   gfx.lineWidth(3)
      .lineColor(color(0,0,0))
      .line(0,0,100,100);
}

the paintContent handler is invoked when content layer of the element needs to be drawn.

This model is close to WM_PAINT based painting in Windows (if to think about DOM elements as windows). The difference is that gfx passed to the handler function uses all current transforms active at the moment of drawing. Including transformations applied by transform and transition CSS properties.

The DOM element supports now three paint "event" handlers:

1. Element.paintContent = function(gfx) {}
2. Element.paintBackground = function(gfx) {}

3. Element.paintForeground = function(gfx) {}

that cover all drawing layers used in DOM element rendering.

Direct2D also required to change drawing on Image’s. Once created the image will become immutable and placed into GPU space. To support this the Image constructor was changed to this:

class Image {
   function this(width,height,paintFunctionRef);
}

Thus to create image with custom rendering you will need to call its constructor providing painter function:

function painter(gfx) { ... }
var myImage = new Image(w,h,painter);

See samples in sciter2.sdk/samples/graphics/

transition:blend(ease,time,…) and friends

transition:blend now supports parametrization, you can provide ease function name, time, etc. In the same way as for atomic CSS properties.

And yet it got transitioning cousins: blend-atop(), scroll-left/top/right/bottom, slide-left/top/right/bottom, slide-over-left/top/right/bottom that use similar principles: to switch to the new state the engine makes snapshots of intial and final states into bitmaps and does transformation of these two bitmaps in various way.

The behavior:frame knows about these transitions and if they defined it applies them when switching content of the frame.

For other use cases the Element.update() function was changed to support optional stateChangerFunction parameter.

function myChangeState()
{
  this.clear();
  this.$content( <p>New content</p> );
  ...
}
someEl.update(myChangeState);

Under the hood the Element.update(changer) does these simple steps:

1. Makes snapshot of intial state of the element;
2. Calls provided changer() function that is expected to make all needed changes for the new state of the element;
3. Makes final state snapshot;
4. Starts the transitioning animation (if it is defined in CSS for the element).

If there is no CSS transition defined for such element the update() simply updates/renders the element in its final state.

The Element.update(changer) feature establishes foundation for other types of effects – expect more of those.

HTML parser

HTML parser was completely redesigned in Sciter2. With the idea to support better new DOM model and HTML5 features. The parser now recognizes these HTML5 elements: SECTION, ARTICLE, ASIDE, HGROUP, HEADER, FOOTER, NAV, MARK, PROGRESS, METER, TIME, FIGURE, DETAILS.

Debug console

All debug output was unified to use the same message format and is grouped to HTML/DOM parser, CSS parse, CSSS! and SCRIPT parser/runtime message clusters.

To setup your own debug console use SciterSetupDebugOutput() function. You also can provide "NOP" function to suppress any debug output  from the engine.

While porting Graphics functionality using Direct2D primitives I’ve tried to implement immediate mode drawing in Sciter.

Problem: as we know HTML5 mandates <canvas> to use off-screen bitmap buffer for drawings. Such model is not transform friendly (scale, rotation) as it involves bitmap transformation. So even when Graphics primitives (line,rectangle,etc.) are vector-ish the result is not a vector but a bitmap. With all consequences.

Solution:

Now you can say something like this:

function myPaintFunction(graphics) {
  ...
}
element.paintContent = myPaintFunction;

Last statement will assign your myPaintFunction to paintContent handler of the DOM element. And your myPaintFunction will be invoked when element will be drawn with graphics object already set for drawing. You can use separate paintContent, paintBackground and paintForeground handlers.

Immediate mode drawing also creates some new opportunities. Imagine that you need to write connector lines between two DOM elements at arbitrary locations. Now you can implement this using these immediate paint feature.

There are plenty of definitions of ‘closure’ term on the web. Most of them quite generic and here is my attempt to define what is the closure under the hood. Hope it will help to someone to understand better the subject.

Technically speaking closure is a data structure that combines reference to function body and non-empty list of call frames active at the moment of declaration.

Closure is created by executing some code that contains declaration of a function that uses variables from outer scopes. In this case VM (virtual machine), while executing the code, has to create not just reference to the function but closure structure – function reference and reference to its current environment[s] – list of call frames that hold used outer variables.

Here is JavaScript example of function that returns closure:

function Foo() {
   var zoo = 2;
   function Bar(p) {
     return zoo + 2; // using 'zoo' variable from outer scope.
   }
   return Bar; // returning inner function reference
}
var bar = Foo(); // here 'bar' contains Bar instance (the closure).
...
alert( bar() );  // invoking the function-closure.

In order Bar function to work it should have reference to outer call frame (the one that contained ‘zoo’ variable).

And here is an example of plain function declaration – function referred by bar uses only its own variables:

var baz = function() {
   var zoo = 2;
   return zoo + 2; // using 'zoo' variable from its own scope.
}

To represent JavaScript closure and plain function we can use following definitions (C++):

  struct Function { // plain JS function
    bytes bytecode; // executable code of the function.
  };
  struct Closure {  // JS closure
    bytes bytecode;        // executable code of the function.
    cframe* cframe_chain;  // callframe chain, used to get/set variables in outer call frames.
  };

If we don’t care too much about memory consumption then we can declare JS function as a generic class that will cover functions and closures in single entity:

  struct Function {
    bytes bytecode;      // executable code of the function.
    cframe* cframe_chain_nullable; // callframe chain, is NULL for plain functions.
  };

For any given JS function compiler can determine is it using outer variables or not. So it can tell if closure creation is required for given function declaration. It makes sense to do such detection as creation of closure is pretty expensive – VM shall move call frames from stack to the heap – convert callframes to GCable objects. If closure is not required by the nature of particular function VM should not create the closure and so heap will not be polluted by unused call frames.

Almost each month I see discussions about character encodings on various software forums. Most of the time these are pure urban myths, legends, hoaxes and rumors. Actually I am surprised how many software developers simply have no idea about the subject.

Here is my attempt to define all these terms in single place and in compact form:

1. windows1251, utf-8, ascii, koi8, etc. are all “transport” encodings of UNICODE code points. Encoding defines format of transmission(or storage) of a text – meaningful sequence of characters of human language(s).
2. UNICODE code point is a 21-bit number – index of a character in UNICODE database (table).
3. Each encoding is characterized by its code unit.
4. Code unit – smallest non-dividable element of the sequence. In most of encodings code unit is a byte – 8-bit number. But there are exceptions. For example: ASCII-7 – 7 bits, UTF-16 – 16 bits, UTF-32 32-bit integer.
5. Encoding can be full – covers whole UNICODE range (e.g. UTF-8) and it can be partial (for example ASCII) – maps subset of UNICODE code points to code units of particular encoding. Strictly speaking any official encoding like ASCII, Windows-1251, etc. is a UNICODE encoding if it has official and/or well known definition of code units mapping to UNICODE.
6. Encoding may have variable number of code units per single UNI-code: UTF-8, UTF-16, GB18030, etc. And there are “fixed” encodings with 1:1 mapping of code unit to UNICODE code points: ASCII, ISO/IEC 8859-1, Windows-1252, and so on.

About UCS-2 and UCS-4.

• UCS-2 – is a 16-bit subset of big UNICODE table. Sometimes is used as a synonym of BMP range – Basic Multilingual Plane.
• UCS-4 – is a full range of UNICODE table (32 bits number where 21 bits are used).
• UCS-2 and UCS-4 are not encodings. These are just names of historic ranges of character codes (UNICODE code points).
• As an examples:
  • JavaScript standard (ECMA-262) defines that String instances represent sequences of UCS-2 (!) codes. So character code in JS is limited by 0xFFFF.
  • In my TIScript string is a UTF-16 sequence so it can operate by full UNICODE range. Thus str.length can be larger than number of characters in string (e.g. for some Far East texts). str[i] will give you number from 0 to 0xFFFF – value of UTF-16 code unit. But if you will write:

      for(var codePoint in "...str..." )
         stdout.printf("%d ", codePoint);
    

    you will get sequence of real UNICODE code points from string.

And that is pretty much it. Not a rocket science, is it?

Preface

Modern Web applications frequently use AJAX kind of client/server interaction. They receive data from the server in pure JSON format. That means instead of generating markup on the server such applications are composing HTML inside the browser (on client side).

Straightforward approach is to use string concatenation spagetti like : "<b>" + data + "</b>".  But this almost always will end up in non-maintainable mess. Real Jedi use markup templates. Typical PHP page is a script with embedded HTML – typical template the gets "instantiated" for the particular GET request/data/user.

PHP or plain old ASP are not only possible template formats. There are a lot of template engines and template languages in the wild. All of them fall into four major groups:

1. Minimalistic, logic-less: {{mustache}} and (probably) PURE;
2. Still simple TDLs but with some simple logic like if/else construcs: jQuery.tmpl() and the KiTE,
3. PHP or ASP alike templates: JavaScript constructs embedded in HTML using <% %> brackets: jQote, John Resig’s Micro Templates, EJS, etc.
4. Group of template engines based on #haml/Ruby ideas – they use special non-HTML markup.

In general: as simple language, less syntax noise it creates – as better. Easier to comprehend and so easier to maintain. The worst case from this perspective is actually PHP (group #3) – mix of two different syntaxes in single source (script and markup) can easily become not readable.  

Speed of template instantiation is on other axis of "templates space". Implementation of PHP alike templating (group #3) is relatively straightforward with JavaScript. Basic idea is to replace all text between
"%> ... some markup... <%"  
by something like
 out += "... some markup...";  
and wrap the template into
 compiled = new Function(transformed_template).

Template instantiation in this case is a matter of calling such function. This approach potentially is as fast as JavaScript itself. But as I said the template source is too "dirty" even in simple cases. I believe that code from this article http://blog.futtta.be/2011/01/18/how-to-do-jquery-templates-with-jqote2/ is a good example of how messy it can be with just few if/else’s.

On other side {{mustache}} templates are pretty clean but current {{mustache}} implementation is extremely slow. According to jsperf.com/dom-vs-innerhtml-based-templating/96 it is 150 times slower than the most effective jQote2. Too bad to be honest.

Another problem with the {{mustache}} is its logic-less nature. I understand the motivation but in reality some simple logic is required. Something like "if fieldA > 10 then render the record one way otherwise in some other".

So I came up with …

The KiTE.

KiTE is lightweight (180 lines of code) and fast JavaScript template engine. It uses template defintion language (TDL) that is close to {{mustache}} but with few additions: conditional sections and custom formatting functions.

Here is an example of KiTE template that emits simple list of contacts:

<ul>
  {{#contacts}}
    <li><b>{{firstName}}</b> <i>{{lastName}}</i></li>
  {{/contacts}}
</ul>

When given by JS data in following format:

{ contacts:
  [ { firstName: "Ernest", lastName:"Hemingway" },
    { firstName: "Scott", lastName:"Fitzgerald" } ]
}

the template will be instantiated as this list:

• Ernest Hemingway
• Scott Fitzgerald

KiTE templates can be placed in

<script type="text/x-kite">
  ...
</script>

sections on the page so they will not polute JavaScript code.

You can use this document http://terrainformatica.com/kite/test-kite.htm to get a feeling of the KiTE templates.

Idea behind KiTE implementation, defintion of TDL and the kite() function are all explained here code.google.com/p/kite/

And the last: the name "KiTE" is acronym of "KiTE is a Template Engine".

Behaviors as an entity is a declarative way to assign/bind scripting methods to DOM elements.

We can think that browsers have following declarations in their default CSS declarations:

input[type=text]   { binding: TextEditorImpl; }
input[type=button] { binding: ButtonImpl; }
select             { binding: SelectImpl; }
...

So when we define <input type="text" /> in our markup we declare that the element will behave as a text editor – it will have set of all needed methods and will generate all associated events.  

It would be nice if in script we would be able to define our own behaviors for classes of DOM elements too.

As an example: blog article may have hyperlinks inside and particular blog engine may require some special behavior/reaction assigned to all hyperlinks in the article. Ideally such declaration should like this:

#content div.article a[href][title]
{
  color: orange; // ui style
  behavior: LinkWithSmartTooltip; // behavioral style
}

In Sciter [1] that is an embeddable HTML/CSS/TIScript engine I have a luxury to step beyond W3C specifications so I’ve implemented the Behaviors in the way as I think they should be:

Behaviors in the Sciter engine

I have added the prototype attribute to my implementation of CSS:

some-CSS-selector
{
   prototype: SomeBehaviorClass [ url(of a script file) ];
   ...
}

When the engine assignes CSS styles to elements it also tries to find class named SomeBehaviorClass. If such class is found then the element gets "subclassed" by the class. Technically the subclassing means that for all DOM elements that satisfy some-CSS-selector both these statements are true:

element instanceof SomeBehaviorClass;
element instanceof Element; // Element is a super class of all DOM elements.

The SomeBehaviorClass looks in TIScript like this:

class SomeBehaviorClass: Behavior
{
   function attached() {} // constructor, sort of
   ...
}

The attached method plays a role of a constructor function in realm of Behaviors. It is called when particular element gets the behavior with this variable referring to the element. All this is I would say is pretty human readable an transparent.

Ok, back to the reality of the Web. Below is my attempt to define similar functionality using jQuery:

Behaviors for conventional browsers,  jQuery extension.

First of all here is my initial implementation of the behavior functionality: jquery-behaviors.js. It is pretty simple – near 75 lines of code.

It allows to declare behaviors on elements by using (a) class DOM attribute like this (purely in markup):

<span class="behavior uix-slider" />
<input class="behavior uix-date" />

and/or by (b) declaration of selector rules in script code:

$.behaviors.selector("ul.ext-list > li", MyExtListItem );

Later case allows to add behaviors non-intrusively – to any existing markup.

The Behaviors implementation above introduces three methods:

1. $.behaviors.add( name, behaviorObj ) – add named behavior that can be used in "a" case above – in class names.
2. $.behaviors.selector( selectorStr, behaviorObj ) – add "CSS selector -> behavior" association – case (b) above.
3. $.fn.behaviors() – that is a plug-in that extends jQuery object wrapping set of elements. It used after calls of DOM mutating methods to assign behaviors to DOM elements:
   $("#update-panel").html("....").behaviors();

The behaviorObj above is the behavior implementation per se. It is a plain JavaScript object that defines set of methods and properties that will be mixed into the DOM element property map.

Here is a typical structure of some behavior named "x-checkbox" (see it’s demo here …) :

$.behaviors.add( "x-checkbox",
{
  $attached: function(params) { ... },
  $value: function(v) { ... },
  $clear: function() { ... }
});

Function $attached here has special meaning – it gets called by the Behaviors engine when the element gets this behavior attached. The params here is a parsed version of the params DOM attribute that allows to parametrize instance of the behavior for the element. For example particular instances of a slider may have different initial settings:

<span id="first" class="behavior uix-slider"
     params="min:0, max:100, values:[15,50]" />
<span id="second" class="behavior uix-slider"
     params="min:10, max:200, value:50" />

Input behaviors, concept of the value. The "x-form" behavior.

In principle there are two distinct types of behaviors:

• input behaviors – behaviors of elements that have a concept of the value. Input elements have at least these two methods:
• method $value(v) – getter/setter of the value.
• method $clear() – clear the value – revert it to the initial, markup declared state.
• UI behaviors – behaviors implementing various UI effects like "click here – expand/collapse there".

The $value and $clear methods are used by the x-form behavior for gathering and setting values of input elements it contains. The x-form element is by itself is an input element (compound one). Its value is a map of name/value pairs – values of standard inputs and elements that have input behaviors attached. Behavior x-form can be assigned to any container that has inputs and call of its method $("#my-form")[0].$value() will give collection of values that are e.g. ready to be send over the AJAX.

The x-form and individual input behaviors may also implement concept of $validate() – that is not implemented yet but planned.

Demos

Here is couple of demonstrations of the approach:

• jq-ui.htm – demo of jQuery-UI widgets wrapped into input and UI behaviors (implementation: jquery-ui-behaviors.js).  Also demonstrates use of x-form to gather/set/clear form data and dynamic html loading with behavior assignment.
• std-behaviors.htm – purely declarative sample. Demonstrates x-checkbox behavior – is a input and UI element that can be bound declaratively with show/hide of "buddy" elements. Uses std-behaviors.js – implementation of the x-form and x-checkbox.

References

1. The Sciter – an embeddable HTML/CSS/Scripting engine;
2. TIScript language – JavaScript++ if you wish. Used in Sciter.
3. Behaviors in Sciter, part I, part II and part III