It's not the size of the dog in the fight, it's the size of the fight in the dog.
To Upgrade G-WAN: (a) overwrite your ./include files and the gwan executable with archive files and then (b) run G-WAN once without -d (daemon mode) to make sure that all your servlets and handlers compile without modifications.
Default MIME type of scripts
G-WAN v4.3.11 accidentally changed the default MIME type of scripts to "text/plain".
G-WAN v4.3.14/Linux: Development Release
- restored "text/html" as the default MIME type of G-WAN scripts
Thanks to IBM for the heads-up.
Minor bug fixes.
G-WAN v4.3.11/Linux: Development Release
- fixed directory listing sorting when folders are small numbers
- fixed the MIME type returned for directory names that contain a dot
Thanks to Ersun and Maurits.
Minor bug fixes.
G-WAN v4.3.10/Linux: Development Release
- enabled the previously defined but filtered PATCH HTTP method
- enabled the GZIP compression of contents using the JSON MIME type
- fixed a JS minifying bug about replace() expressions using '#' characters
Thanks to Anders, Ersun and Frank for the feedback.
Amazon EC2 instance workaround, host IP addresses count fix, CGI, URI
Workaround for hosts reporting a zero count of CPU Cores (like some Amazon instances) and a bug fix for the machines which have many IP addresses. Server-size CGI environment variables are now re-enabled and RESTful URI parameters are processed correctly, even for mixed modes (like /?argv&lg=en_UK&file=about/company/&user=john).
G-WAN v4.3.8/Linux: Development Release
- added a check for the Amazon instances which report zero CPU Core
- fixed a bug related to the initial count of a large volume of IP addresses
- fixed the parsing of URI parameters relying on both '&' and '/' characters
- restored the exportation of CGI environment variables for PHP, Perl, etc.
- added the ImageMagick imgsz.c example, to resize & watermark pictures
Thanks to Ersun, Olivier, Navin and Michael for the ideas and reports.
CGI Scripted Servlets Optimization, Disks usage levels and File System type
Better tuned the asynchronous Lorez Waterwheel to provide more 'air' for heavy works. Logged more information about the local host's disks (total/used/free space levels and File System type).
G-WAN v4.3.1/Linux: Development Release
- better tuned the internal threading scheduler and dispatcher
- logged total/used/free space levels and File System types of local disks
Thanks to Michael for the heads-up and to Laura for the suggestions.
Ubuntu 12.04/ext4 daemon-mode (-d:www-data) workaround
Removed the "server started" static string previously stored in the /logs/gwan.log file to indicate the server stage. Other steps (all of which are dynamically generated strings) are still logged: their do not pose any problem.
G-WAN v4.2.28/Linux: Development Release
- fixed the ./gwan -d:www-data daemon-mode on Ubuntu 12.04/ext4 by removing a string from the server log file
That's the only change – and now G-WAN can be used in daemon mode (without immediately crashing) on Ubuntu 12.04 64-bit. Other Linux distributions were not affected by the static string logged in gwan.log since G-WAN v1 in year 2009.
Some users reported that G-WAN works fine on their their AMD CPUs. Others report that C scripts can't be used on their AMD CPUs. If you are using AMD CPUs, please send us your /logs/gwan.log file in both cases so we can try to find a pattern.
Ubuntu 12.04/ext4 workaround, CSS/JS minifying fix
This release provides a simplified BASIC authorization example, and more defensive code for diverse platforms as well as bug fixes.
Pointless on prior versions like Ubuntu 10.10/ext3, the Ubuntu 12.04/ext4 workaround is very important as it fixes an issue that made G-WAN return 404 errors for 1/3 of the requests (static and dynamic)... only when the path of the gwan executable had a given 'length modulo x' (hence the problem seen with listeners on port 80, but not seen on port 8080).
G-WAN v4.2.27/Linux: Development Release
- added a workaround for a possible 0:00 issue triggered because of a missing resource
- fixed a CSS-only problem and a JS minifying issue, both leading to incorrect results
- fixed a system-dependent issue triggered by the ./gwan path length on Ubuntu 12.04/ext4
Thanks for the very high-quality feedback of Brennan, Dawid, Maurits and Tomas.
TCP/IP Firewall, large POST entities, BASIC authorization example
The firewall feature lets you dynamically blacklist all traffic coming from abusers, either on a per IP address basis or from a per CIDR network.
Sure, there are dedicated tools (IDS) that detect and fire IP-based blocking, but there are times when the server and the engineer know better how their apps work than automated tools. Further:
"A chain is only as strong as its weakest link." (Charles A. Lindbergh)
You will have to run a separate ./gwan daemon in root mode to modify the firewall rules (this is a security imposed by Linux). It is recommended to make it listen on 127.0.0.1 ONLY (on the port of your choice). From your running app. server scripts, connect to this ./gwan firewall instance to pass your orders. For an extra security layer, you can use a secret password (or encryption) between the two server instances.
G-WAN v4.2.19/Linux: Development Release
- added the example auth_basic.c to demonstrate BASIC user authorization
- added a TCP/IP firewall API that can be used from G-WAN servlets and handlers
- fixed a large file uploads glitch and improved entity.c and entity_size.c
Some users reported that they could access www documents but not csp scripts. Further investigations revealed that they are using AMD CPUs. As we don't have any AMD CPU among the 21 computers of our office we can't find what's wrong. Any insight from experienced AMD CPU users is welcome. This problem appeared with v4.2.13.
Streaming, Workaround for VPS
Updated the stream3.c example and fixed a glitch with long streaming. Also added more defensive code as some hypervisors make pthread_join() fail, creating a recurring G-WAN silent exit, notably visible during the log rotation process.
G-WAN v4.2.13/Linux: Development Release
This is a minor release, useful for those either at the cutting-edge (streaming) or those using G-WAN on virtualized servers.
Custom HTTP headers in HTTP errors, File System bug for sendfile
The same issue fixed in v4.2.5 for the general case remained for the sendfile() branch. This is now fixed.
G-WAN v4.2.7/Linux: Development Release
- added support for http_headers(HEAD_ADD) in HTTP Errors (thanks Paulo)
- added the socket and a summary of the server response in the trace file (./gwan -t)
- fixed the same previous version time File System issue but this time for sendfile
Exception made of the experimental features, this release should be close to the production status. We will have more time now to document the features developed in late 2012.
BTW, G-WAN is almost twice slower on Ubuntu 12.04 64-bit than on Ubuntu 10.10 64-bit (the previous LTS).
File System issue, error.log: HTTP Status Code 444, Source Code Audit
stat() file times are supposed to return nanoseconds when available. Since G-WAN's internal time resolution is the microsecond, G-WAN tested if nanoseconds were non-zero and used them when available. This created problems on those file systems where nanoseconds are reported in micro-seconds. That's why Ubuntu 10.10 ext3 worked fine while Ubuntu 12.04 ext4 sometimes (when copied from an ext3 FS?) reported empty files: dwarfed file times (by a factor 1000x) made G-WAN send misplaced "304 Not Modified" replies and use 1970 log file or directory listing stamps.
G-WAN v4.2.5/Linux: Development Release
- added some HTTP Status Codes for error.log, including Nginx's "444 No Response" (used for timeouts)
- fixed a bug preventing incremental heap growth to merge large blocks of memory (that was not visible externally)
- audited G-WAN's source code with static and dynamic analysis tools (gazillions of false alerts, not one single issue)
On July 2013, G-WAN will start its 5th year online. It now counts 110,670
lines of code (~40% of comments).
In comparison, Nginx 1.3.11 (which starts its 11th year), counts 133,808 lines of code (and ~4% of comments).
As an application server, G-WAN supports scripts in 15 programming languages, offers many more features... and G-WAN does this all in half the amount of code required by Nginx.
I don't know if that's because G-WAN is closed-source, but it has 10x more comments than Nginx's "open-source" code.
When comments are trivial or redundant, they are better avoided. But when they let you FIND the portion of code that you are looking for (either to use, to enhance or to fix it) then relevant comments are saving you a lot of time.
Almost all Nginx comments are made of (redundant) copyright notices, making it harder for users to read, enhance or fix this code which, even after 11 years of exitence, still requires constant patching exposing users to new security holes.
|server||birth||files||indentation||blank lines||comment lines||source code lines||total lines|
These are objective metrics, and they clearly illustrate G-WAN's value:
- number of lines of code
- number of lines of comments
- features per line of code
- speed of features
- scalability of features
- number ofengineers writing the code
- total time required to write all the code.
Which team would you hire to write code if your company had to pay the bills and to sell the resulting apps?
Build optimization issues
The GCC compiler did not like some parts of the G-WAN v4.24 code. This build reverts to a less sophisticated codebase that works as expected (the failing code works fine with -O0 but fails with -Os), removing the bugs observed in v4.24 (empty files, empty dates) generated by the compiler.
G-WAN v4.1.25/Linux: Development Release
- recompiled v4.25 with less sophisticated code (which compiles as expected)
Thanks Olivier for the quick and helpful reports.
Virtualizer OOM Patch, C++/D/Objective-C maintenance/handler scripts
Some more defensive code for users who use a virtualizer (Xen, VMWare, etc.) configured to confine G-WAN in an environment having (a) many CPU Cores, (b) plenty of scripts, (c) JVM and Mono virtual machines loaded – and less than 1GB RAM.
G-WAN was interrupted at startup with an "out of memory" error (avoiding the less intuitive silent Linux OOM kill-switch).
We now make it clearer what the cause of trouble is by checking this condition before it is met deeper by the running code.
G-WAN v4.1.24/Linux: Development Release
- added experimental (almost completed) servlet streaming support
- added a patch for virtualizers that setup very low amounts of RAM
- added support for C++/D/Obj-C handlers and maintenance scripts
- fixed a bug preventing gc_init() API function call from working
- added a check to let xbuf_repl() run anyway on empty xbuffers
- fixed a compiler optimization which was breaking the getns() call
- fixed an initialization that could lead to crashing maintenance scripts
Thanks Are, Daniel, Karim, Kin, Michael, Michal, Mike, Olivier, Richard, Sergei, Thomas J., Thomas M., and Tomas.
Virtualized CPU: [0 CPU, 1 Core], Directory Listings: missing CSS
A minor bug-fix release for the brain-damaged virtualizers who proudly report... zero CPU and one Core in their virtual machine.
By the way, this elucidates the "Floating Point error" mystery as this division by zero was due to the bright mind of those who break an OS what works pretty nicely – without their assistance.
After glibc, virtualizers are the next party-spoilers: that's the only remaining obstacle between G-WAN and the Linux kernel.
G-WAN v4.1.18/Linux: Development Release
- added a patch for virtualizers that give a bogus CPU count
- replaced the www/imgs/dl.css file by a working file
- modified the /gzip policy to better handle small files
Thanks Akos, Kiswono, Michael and Paulo.
Linux-Wide (and BSD) Portability
With v3.12, we learned that glibc is better left "as is": its arcane internal structures and complex compatibility versioning rules break with the wrappers we wrote in an attempt to be safer against the major incompatibilities of... glibc with itself.
If you link with an old glibc, newer glibcs can run your apps. But if users haven't the latest glibc, unless you maintain (glibc-compliant!) patches for all past glibc changes then your program will be vulnerable to glibc's security holes, crashes, etc. This is why v3.12 tried to provide alternative calls for some fatal glibc issues.
Having developers use a recent glibc can be done, but things may break at any time on remote servers, either hosted, at customers' Data Centers or even embedded – where online OS updates are not an option. Further, when statically linked, glibc still requires at runtime a copy of the exact shared libraries used during linkage (defeating the purpose of static linking).
To let users (a) run G-WAN on any Linux distribution/version and (b) load libraries to add features to handlers or servlets we have had no choice but to (c) remove glibc from the equation.
We also have had to design two versions of G-WAN, each being linked differently:
- dynamically (for development, with normal /usr/lib libraries and programming language runtimes)
- statically (for deployments, where your G-WAN scripts are statically linked to the gwan executable)
The glibc-free dynamically linked G-WAN version works like a charm (smaller and faster). More work is needed to support C++ humongus libraries. As all libraries must use the G-WAN glibc-free runtime, an online service will probably link people's static object code and libraries with the G-WAN runtime (like others, we will also provide our own stable Linux distribution).
The glibc-free statically linked version is even smaller and faster. Some users asked us how to deploy G-WAN applications without prodiving the source code of their scripts to their users – this is possible now. This version also protects applications against accidental (or malicious) tampering of the shared runtime used by all dynamically linked apps (Linux's default).
For both versions of G-WAN, where portability, stability, footprint and performance all matter, that means:
- universal portability on all Linux distributions (probably also on BSD)
- increased security (using different system library will not affect G-WAN)
- better and predictable efficiency (as well as lower memory footprints).
And if you believe that the expected gains of a smaller footprint are marginal then think again:
gcc -s -Os hello.c -static 883 KiB (glibc) gcc -s -Os hello.c -static 5 KiB (g-wan)
The glibc bloat is hidden when you link dynamicaly... but it weights. Especially at startup, on low-end hardware, or under load. This will lead to dramatic improvements on embedded systems (routers, cellphones, etc.) and performance clusters.
That G-WAN evolution will clearly benefit to developers and end-users (no more platform-dependant issues, no more mystery hunts, much lower support costs). The libraries used by G-WAN scripts as well as the scripts will be linked to G-WAN, having one single file: a customized G-WAN executable.
This personalization capability can be mis-used so it will be limited to registered users. This service is offered to help G-WAN users, this is not the core of our business so the goal here is merely to cover our operating costs.
Unregistered users can still use glibc-based G-WAN (a freeware) for free. Most of the time, it will run fine if used in harmony with the way Linux is distributed: on the forever boiling, up-to-date, fully-patched installations.
We did not plan to replace the system runtime: this just came out as a dire necessity for our own G-WAN-based applications: how can we market apps which fail to run at the customer's place just because a library was modified by a third-party?
G-WAN v4.1.17/Linux: Development Release
- added a far more detailled CPU and Cores detection, seen in gwan.log
- modified throttle_reply() to avoid using more recent kernel versions
- allowed multiple/mixed HEAD_ADD/HEAD_AFTER http_header() entries
- fixed many erratic problems by no longer using the glibc wrappers and syscalls
- removed all "apt-get install" invites as packages change constantly
- fixed failures to use the C#/Mono runtime (see the FAQ to install C#/Mono)
- replaced the 64-bit program by the 32-bit program in the 32-bit archive
Kernel Compatibility of glibc-free G-WAN versions
With epoll support (kernel 2.6.0: Dec 18, 2003 or more recent) G-WAN should able to run and to use multicore CPUs.
Thanks all for the perilous exercise of reporting erratic errors with no obvious pattern to incriminate in established G-WAN Core code that otherwise worked fine during years. Thanks Paulo for http_header().
Don't forget to provide the /gwan/logs/gwan.log file with bug reports (it helps to identify problems). And if you are using a virtualizer then please let us know it's name and version number.