Red Programming Language
July 17, 2017
0.6.3: macOS GUI backend
830 commits, closes 86 issues, and brings a large number of new features. The main one is official support for the macOS GUI, through a new backend for the View engine.
For readers not familiar with our GUI system, you can get a quick overview in a previous article.
- pluggable backends (full support for Windows and prototype for GTK and Android)
- full 2D vector graphics support (a large subset of SVG)
- direct reactiveprogramming support (not FRP style for now)
- a declarative DSL for GUI building and layout management
- a declarative DSL for 2D graphics
The macOS backend supports all the same features as the Windows backend, except the following, yet to be implemented:
- fill-pen pattern , fill-pen bitmap and diamond gradient in Draw.
- matrix transformations on pen and fill-pen in Draw.
The GUI console for macOS will be available in the 0.6.4 branch, so is not part of this release. The View module, though, is compiled in the CLI console by default on macOS, so no show-stopper there.
A tiny demo script (hopefully, it should get us an office at Station F, right? ;-)):
A small reactive text size measuring tool (one of our test scripts):
This will produce a tiger.app bundle on the Desktop:
- color-backgrounds : color the background of some colorless faces to match their parent’s color
- color-tabpanel-children : Like color-backgrounds , but tab-panel specific
- OK-Cancel : buttons ordering rule, puts Cancel/Delete/Remove buttons last
- adjust-buttons : use standard button sub-classes when buttons are narrow enough
- capitalize : capitalize widget text according to macOS guidelines
- Cancel-OK : buttons ordering rule, puts Ok/Save/Apply buttons last
To give you a quick grasp about why that feature matters and what it is capable of, here is a simple example, which leverages the buttons ordering and capitalization rules:
You can see, side-by-side, the macOS and Windows generated forms. Notice the button text and ordering, yes, they differ from the source code! The GUI rules have ensured that:
- the buttons are ordered according to each platform’s guidelines, “Ok” last on macOS, “Cancel” last on Windows.
- the button’s labels are properly capitalized on macOS.
This small example just demonstrates the concept, but actually, there is no limit on how much it could alter the UI and how far it could go with the post-processing.
You can also remove rules selectively, by modifying the content of the following lists:
This allows you total control where needed, but also helps you conform to UI guidelines with no effort, saving everyone time. Not only do you not have to tweak your code for each platform when you write it, when a new platform is added, you won’t have to change your code to support it. In a world where getting one detail wrong may keep you out of an app store, or prevent you from being a “certified” app, this is huge. It should also be possible to use the rule engine to write a guideline linter, where you could get a report of what rules will be applied to your VID code on each platform. By “report”, we don’t mean just a text listing, but it could highlight elements in the UI that were processed by rules. Having a system that helps you is great. Having a system that tells you how it helped you is even better.
Other changes in 0.6.3
Math and other operations are fully supported:
Red also adds three new accessors:
- /timezone : changes the zone and adjusts time accordingly (UTC-invariant)
- /week : gets/sets the week of the year, starting on Sunday.
- /isoweek : gets/sets the ISO week of the year, starting on Monday.
Conversions to/from Unix epoch time are also built-in:
- delete: deletes a file or an empty folder.
- size? : returns the size in bytes of a file.
The do native now accepts a URL argument. For example:
Run-time error reports now include a compact call stack trace:
- system/options/cache : points to the cache folder used by the Red toolchain.
- system/options/thru-cache : points to the cache folder used by *-thru functions.
- system/catalog/accessors : Path accessors available, per datatype.
The browse native can now work on macOS (it opens the default browser on a given URL).
- Now is now fully operational and can return current date and time.
- Wait now accepts a time! value argument
- Improved make and to url! construction when the spec argument is a block.
- Auto-detect older Intel CPU on Windows platform when building the console.
- Even? and odd? now support time! values as argument.
- R/S compiler can now output a function address map for verbosity >= 3
- Improved min and max natives support for pair! and tuple!( now min/max is applied per dimension to pair! and tuple! , and can work with a number! as second argument).
- -t compilation option does not force release mode anymore if target is the same OS.
- Source file name in now displayed on syntax errors at compile-time.
- redBinary() : constructs a binary! value from an external byte array.
- redImage() : constructs an image! value from various possible external byte arrays.
- redGetField() : gets the value of an object’s word.
- redSetField() : sets the value of an object’s word.
- Set the focus on the face where you want to simulate an event (using set-focus ).
- Trigger an event (using do-event ).
- Test if the result of the event conforms to your expectations.
Here is an extract from a short test script:
Using this test backend requires setting some options in the header (see the linked script’s header), and compilation in release mode.
The first version applies the actions block to all faces, while the second one ( /with ) will select only faces matching the conditions block (which needs to return a true value to act on the current face). Both the actions and conditions blocks have an implicit face word defined, which refers to the current face in the tree. For example:
Here is a more sophisticated example using foreach-face to dynamically localize labels and adjust buttons size and position accordingly, so that they fit the text nicely:
You can also detect window resizing events, and use the current window size to resize and position faces dynamically.
- Lit-words are now accepted in flags facet blocks.
- New Shape dialect command: close .
- Changed default background color to white in tab-panel (Windows).
- Added system/view/metrics (used mostly by VID, for more accurate sizing/positioning).
- Added accelerated? option in face, when set to true, for faster/smoother face rendering (only base faces). Z-ordering is then only honored between accelerated faces.
- When a button has the focus, pressing enter key will trigger a click event.
- Renamed enable? facet to the more correct enabled? .
- New function: set-focus (sets the focus on a given face, important for GUI testing).
- New class option in face/options , allows setting a sub-class of a native control.
- text-list now accepts a data keyword and any-string! values as input. This lets you to feed the list with values from VID. For example:
The above script is just a single expression using two DSLs. I let you meditate on that. 😉
VID has received many enhancements too. The most significant are:
- more accurate sizing and positioning of native widgets using OS metrics
- addition of alignment control, extending across/below layout modes.
So, it is now possible to add optional alignment keywords for the two linear layout modes:
- across: top, middle, bottom
- below: left, center, right
Those keywords can also follow the return command, if changing the alignment mode of the next row/column is required. The defaults are top and left . Here is a short example:
- New on-created event, triggered just after a face has been shown for the first time.
- New strike option for faces: sets the strikethrough font style.
- New init [<body>] styling option allows you to define a block of code to run when the style is instantiated.
- later option added to react keyword.
- Allows data and at keyword argument to be an arbitrary Red expression (see the above text-list example).
- More robust face options parsing.
- The default actor for h1 to h5 and text widgets is now on-down .
- VID now saves the style name for each face in face/options/style .
Passing structs by value is sadly not formally specified in C language, resulting in incompatible compiler-specific ABIs. Moreover, those ABIs are not documented, or only partially, so gathering all the right information was painful and time consuming. Just to illustrate how bad the situation is, after spending days researching all the various C compilers ABI, I was able to answer a stackoverflow question about returning C structs by value, which was left unanswered for 6 years. I might write a synthetic article about those C ABIs someday, from my notes, as I could not find such documentation online. Anyway, the hard work has been done for you. Red/System now implements the most common ABI for each platform:
- Windows: Microsoft Visual C ABI
- Linux: gcc/Clang ABI (with support for ARM-specific ABI)
- macOS: Clang ABI
In addition, more stack-oriented features are now supported:
- system/stack/allocate : allocates a buffer on the native stack.
- system/stack/free : frees a buffer on a the native stack.
- New use keyword: defines a local namespace in a function.
The new use feature allows you to create local variables anywhere in a function body. It is useful for splitting long functions and creating local variables in macros.
- fixes and improvements to soft integer division routine.
- fixes non-passing unit tests on floats.
- better AAPCS conformity and various bug fixes.
- improved accuracy of overflow detection in multiplication.
- improved auto-completion, now completes the longest common substring.
- auto-detect older Intel CPU (non-SSE3) on Windows platform when building console.
- system/console/size now provides the size of the console in columns and rows
March 26, 2017
0.6.2: LibRed and Macros
- Macros and preprocessor support
- Fast compilation using libRedRT
- LibRed for embedding Red anywhere
Development and release modes for compilation
- development: ( -c ) only user code is compiled (new mode).
- release: ( -r ) both user code and Red runtime library are compiled together.
Another benchmark is compiling the Red tests suite (
18000 tests), unified way combines the tests into the minimum number of compilation units, while split way compiles each test file separately:
Such performance results in significant daily productivity gains, both for the Red team, and Red users. It was worth the time and effort it took to properly convert the runtime library into a shared library. Though, the full support for modular compilation will come in 0.8.0, which will result in drastic cuts for the compilation time in release mode too.
- Red/System code does not contain any call to Red runtime library.
- Red/System code contains one or several calls to Red runtime library.
In the first case, nothing needs to be done, the usage rules described above apply.
Embedding Red using libRed
A libRed graphic HelloWorld in C:
All the code for this demo fits in a single page of VB code! You can get the required files from here. In that same Excel file, you will find two other simple examples of integration of Red with Excel sheets, which look like this:
That will build libRed with the cdecl ABI, suitable for integration with any C-compatible host. For VBA and MS Office, the stdcall ABI is required, which is achieved using:
The building process will also result in creating a libRed/ directory locally, expanding some extra files required for linking libRed properly.
- New datatypes: tag!, email!
- New action: to
- New natives: as, call
- New functions: tag?, email?, hex-to-rgb, sqrt, class-of, face?, distance?, expand-directives, to-*, rejoin
- Adds integers auto-promotion to floats on loading and on some integer operations.
If you need to preprocess the input to LOAD, you can now do it easily by plugging a function into system/lexer/pre-load . This feature is mostly meant for pre-processing the console’s input, though it could also be used for changing some Red syntactic rules. For example:
Another usage could be to translate words on-the-fly in the console:
- a block of successfully loaded values.
- the input string at the position where the lexer failed.
- an error! value (or none! value if the tail of the string has been reached).
Command-line argument processing has been mostly rewritten to provide a consistent experience across platforms and type of binaries (red executables, console executables, compiled user scripts and, to a lesser extent, Red/System executables). The new features are:
- system/options/script refers to the script name ( string! or none! ).
- system/options/args refers to a list of tokenized arguments ( block! or none! ).
- system/script/args refers to the original command-line ( string! or none! ).
- full Unicode support for red executable’s arguments on Windows.
- single-quoted arguments are accepted on all platforms (same as double quotes)
- multiple nested quotes are treated as just one level of quoting when splitting the command-line.
Datatypes conversions are now fully supported in Red! The to action is now officially supported, and make action has been completed. The usual to-<type> helper functions you can find in Rebol, are also now defined. All the conversion rules are documented in an Excel matrix for now.
- box , h1 to h5 styles added to VID.
- Colors in VID can now be specified as hex values, using #rgb or #rrggbb formats.
- Adds support for no-border flag to area and field face types.
- Adds now option to rate keyword in VID, to fire on-time actor at once.
- The wheel event and on-wheel actor are now available for handling mouse wheel events.
- Default tab size for area changed to 4 spaces.
- View now uses DirectWrite to draw text in base face (except WindowsXP).
- Better handling of default fonts.
- Enhanced GUI console, with new settings window with pre-selected colors picker.
- A hint text can now be specified in a field options block and a hint keyword has been added to VID. For example:
- Matrix operations support: matrix, invert-matrix, reset-matrix, matrix-order, push, rotate, scale, translate, skew, transform .
- A new clip command is available for defining a clipped drawing/filling region.
- A Shape sub-dialect has been added for more complex shapes drawing and filling.
- A crop option is now available for image command.
- pen and fill-pen have been vastly extended to allow drawing and filling with gradients, patterns, arbitrary shapes and images. Have a look at some of the new capabilities (source):
- insert command now also support position argument (like change ).
- added pick option to keep , so user can control how keep captures the matched input:
- keep collects matched values as a series if many, or as a value if only one.
- keep pick collects all the matched values separately in a block.
- keep copy <word> collects all the matched values as a single series (of same type as input).
- Support for float! / float32! conversions from/to integer! .
- Adds system/cpu/overflow? field for reading CPU’s integer overflow state.
- Adds support for importing variables from shared libraries.
- Allows loading of libraries from current folder and PATH environment variable on macOS.
- Now #call directive supports function calls with refinements.
- Default ABI for exported functions is now settable through export-ABI entry in config object.
- Renamed log and log10 imports from libC, respectively to log-2 and log-10 .
- Now size? accepts a context path argument.
- Improved error reporting for “missing argument” errors.
- A prototype Red/.Net bridge has been introduced.
- New –config [. ] command-line option, for passing a block of compiler settings.
- Added -s and –show-expanded command-line options to output expanded version of compiled source code.
- Added /target option to react? .
- Added /seek and /lines option to write .
- Added /expand refinement to do for preprocessor invocation.
- Added math function for evaluating code using math precedence rules.
- CTRL-L key combination can now clear the GUI console’s screen.
- Checksum function can now trigger object on-deep-change event.
- Now keep returns its argument ( collect function).
- Added temporary rejoin function.
- Added ‘ class reflective property to objects.
- Added class-of accessor (only objects).
- Nicer handling of line breaks in molded image! binary buffer.
- Now #include is converted to do in interpreted code (using macros).
- Zero? function is now a native and supports time! values.
Also, more than 150 issues have been fixed (and wishes granted) during the last months, 22 issues marked as bugs are left open.
December 17, 2016
Incursion into “explorable explanations” lands
280 LOC) but also great fun! Here is the result:
You can, of course, try it yourself by using the latest build from the master (or macGUI) branch or, preferably, one of the prebuilt consoles:
- Ballots demo source on github or zipped archive (171KB).
- Windows prebuilt console (273KB).
- macOS prebuilt console (256KB).
- Original web version from Nicky on github.
Put the console executable in the source code folder, run it and from the prompt, just type:
For Linux folks, sorry guys, the GUI support is not yet ready for prime time, though in the meantime, you could use Wine, Red GUI usually runs just fine on it.
- You need to ` chmod +x` the binary before running it from the terminal.
- When dragging a face quickly, it can lag behind. The macOS GUI backend is still in a development branch, and still need some work to optimize animation latencies.
December 2, 2016
Entering the World of Macros
when evaluated will output:
When the macro is expanded by the preprocessor, the above source code will result in:
This kind of simple macro is called a named macro in Red’s jargon. Once the preprocessor has finished his work, no macro definition or call, nor any preprocessor directive remains in the source code (compiler directives defined as issue! values, are still there though).
As you can see both func and function constructors are accepted for declaring a macro.
Now, let’s get back to the first example and expand (no pun intended) on it:
So macros can freely call other macros, the same way functions would. This is possible because macros are running in a special (hidden) context, which can be accessed and extended (indirectly) by the user using the #do directive:
The #do directive accepts arbitrary code, so local words and functions can be defined there freely, and accessed from macros. This gives a pretty powerful programming layer for the preprocessor, though, it is possible to go even further.
would result in:
In that example, the macro gets called each time a number! followed by the word! KB is encountered in the source code. They will then both get replaced by the macro returned value. The arguments of pattern-matching macros are always the same: one reference to the starting position ( s ) of the matched pattern, and one reference to the ending position ( e ).
would result in:
would result in:
This pattern-matching macro relies on the manual mode ( [manual] attribute), where the replacement is done by user code and the returned value needs to be the position in the source code where the expansion process resumes. In this case, loop is also an existing function, so when the argument is an integer, the source is left untouched. With a normal macro, the resuming point would have been the beginning of the pattern, resulting in an infinite loop (again, no pun intended). 😉
Math expressions folding macro
would result in:
HTML validating macro
The embedded HTML above has an error, the <b> tag is not closed. This macro will catch that error and report it properly during the compilation.
Note: I tried to keep this example short, so it ends up as a BASIC source code compiler to Red, but fed to a custom interpreter. Fully compiling that DSL to Red code would be possible, but would require more complex constructs, in order to deal with a GOTO able to arbitrarily jump anywhere, and that is beyond the scope of this article.
- As there is no difference in Red between code and data, both can be transformed by the same macros, which is not always desirable. Some mechanisms for limiting the application scope exist in the preprocessor, though there is no guarantee they can cover all use-cases. Stay alert, especially with pattern-matching macros.
- It is not always easy to reason about and debug macros, unless you have some existing experience. I would suggest not using them until you have a good grasp of Red’s toolchain, semantics and fundamental concepts (like homoiconicity).
- As the Red toolchain is currently run by a Rebol2 interpreter, Rebol is running the compile-time macros, so keep that in mind when writing them. If you want them to run equally well on the interpreter, you need to use only the common subset between Rebol2 and Red. Sooner or later we should move compile-time preprocessing to use a Red engine (thanks to libRed), so this concern is temporary.
Last but not least, for those wondering about syntactic macros (aka readers macros) inclusion in Red, as for AST-macros, they are not strictly necessary, as the Parse DSL already provides us a powerful tool to implement pre-load-time processing. Though they could still bring some extra benefits (like embedding the processing logic within the source code), but could also go against the Red-as-data-format principle, or wreak havoc in IDE support (like messing up syntax coloring and step by step debugging). We need more time to go through each concern and see how to deal with them before adding such a feature.