Eclipse CDT Linux Howto

UPDATE 5 March 2013: I confirm, Juno CDT has done few improvement.. but when doing a real job, restarting eclipse once every hour isn't handful.. and it is still too much slow (this is my opinion after few days of real use). My opinion is that Eclipse CDT Juno is not the best solution for working with C/C++ and being productive.

UPDATE 3 March 2013: Eclipse CDT performance looks better after the last IDE update. After 3 hours Xorg still raises cpu usage and things slow down but it is far far better.


Eclipse CDT looks a great c/c++ IDE but, with default settings, is unusable is very slow in Linux; The main issues about default configuration, are:

• memory consumption, gc slowness
• Xorg (linux) cpu usage, more slowness
• very very slow
• slowness
• bugs which causes crashes

Using Eclipse CDT is a pain and it looks unusable (Intel Q9300@2500 Core2Quad, 8GB and SSD);I have fine-tuned the default configuration accordingly to this guide and other stackoverflow answers in order to solve these issues.

Let's fix it!

Edit eclipse.ini and raise the VM memory settings, these are mine (I have 8GB of ram so these values are a bit exaggerated, by default I am allocating 2 Gbytes for eclipse):

-Xss2m
-Xms1024m
-Xmx4096m
-XX:MaxPermSize=1024m
-XX:PermSize=1024m

Raising these values may help GC (Garbage collector). Perhaps, I have noted less ide assertions and errors.

Eclipse linux Theme support is a useless mess, GTK+-Qt is buggy and has lots of leaks which causes Xorg to eat up all your cpu, disable the GTK-Qt theme support.

Execute eclipse, go into Windows->Preferences->General->Appearance and set the Theme to Classic (If you know a better way please give me feedback) 

There is an old bug which causes Eclipse CDT to crash suddenly, add this line (vm configuration) in eclipse.ini :

-XX:-UseCompressedOops 

Tune the CDT settings in order to be less aggressive (CDT has to self parse the sources), I have adopted only few changes, let's quote form guide (I suggest to read ALL the guide, it is useful when working on big projects):

Whenever you create a new workspace for a Mozilla source tree, you should be sure to turn off the following two settings in the workspace preferences (Window > Preferences, or Eclipse > Preferences) before creating a project in that workspace:

• in "General > Workspace", disable "Build automatically"
• in "C/C++ > Indexer", disable "Automatically update the index"

Turning off automatic indexing prevents the CPU intensive indexer from running at various stages during the steps below before we're ready.

Warning, thanks to this change, while developing, you need sometime to trigger indexing manually (right click on the project, Index->Rebuild).

You can also configure your project properties in order to enable multicore compilation: Project Properties_> C/C++ Build -> Behavior -> Enable Parallel builds

Conclusion

The fine tuned Eclipse CDT  looks usable, there are no hangs; The IDE still slow down during the programming sessions and became very slow in 1 or 2 hour.


UPDATE: I feel like my how-to is a workaround, in fact it seems there are issues with Eclipse UI performance in Juno, let's look here

gwt-java-benchmarks released

Hi,

 I've just released gwt-java-benchmarks code under GPLv2 license.

I think this is a fair quality code, not Enterprise grade ;-) but it's done and just works.

I am going to tune the sieve benchmark with high mem configuration in the future but it is better to remove at the moment those values from your csv output.

Hope useful.

P.s: I'm looking for suggestions about SVN repository structure.

At the moment the library code is simply cut&pasted in 2 sub projects.. it is the same code, so I hope there si a better way to share code between NetBeans projects; at the moment this is the most viable solution.

Have fun

EDIT: I have tuned the sieve benchmark, problem fixed.

GWT Benchmarks: GWT+JsVM vs JavaVM

What is GWT?

GWT (Google Web Toolkit) is a tool for compiling Java into JavaScript code; with GWT you can write both your web application and your server side code using the Java language.

 

What about GWT performance?

I have worked with GWT in the recent years and a simple question has arisen:

"How does the same Java code perform when running inside the Java virtual machine VS running inside the browser?"

I have decided to do a bunch experiments and write this article after taking in account these facts:

• GWT developers state that the generated JavaScript code may be better than the handwritten one
• V8 JavaScript Engine and the other JavaScript engines have been highly improved in the recent years

It also makes sense to measure the perormance of GWT applications running on different browsers, Firefox, Opera, Chrome, Explorer in order to see who sucks and who rules.  

I think that these results may be useful from a distribuited High Perfomance Computing perspective (via web client).

 

 ..but what is performance?

I'm looking to numeric and data crunching performance; input/output, graphics and multimedia performace are out of topic.

I have measured execution time of the same code compiled with GWT, running inside the browser JavaScript VM(JsVM), and running "natively" inside the Java SE7 Virtual Machine (JavaVM).

Little execution time means high performace, it's quite simple. 

 

The Benchmark suite (Internals)

I have developed a little suite (EDIT: sources released) in order to run a bunch of deterministic single threaded benchmarks covering: strings crunching, numerical crunching and data manipulation; The suite performs these benchmarks after a warm up.

The warm up has been configured in order to make sure that all the benchmark code is JITted first.

To be clear, the JsVM compiles the JavaScript after a lot of execution cycles (e.g: 11000) but JavaVM  is going to compile the  bytecode to native code in few cycles.

After the warm up, the benchmark is executed lot of times and the overall execution time is recorded, it's the same approach than taking an average execution time.

The GWT and native benchmark suites share the same Java code and benchmark configurations, only the Main classes are different.

Here is the list and description of the performed benchmarks:

• CollectionSortBenchmark, Test Java collection sort and shuffle, manipulation.
• FFTBenchmaark, Computes FFT's of complex, double precision data, GSL code taken from scimark
• RegexBenchMark, apply a Regex (Javascript compatible) to a text
• SieveBenchmark, Computes prime numbers into a given numeric range
• StringCrunchBenchmark, String and StringBuilder manipulation, append, delete  

These benchmarks have been performed twice with different configurations, one time using little data sets, and a second time on a huge data sets (big arrays, big lists and so on).

This is intended to track the JsVM and JavaVM behavior when playing with memory demanding applications, yes, this looks like an Good&Old Garbage Collector stress test. ;-)
 

The Benchmark Setup

The suite has been executed on an asus K51 notebook equipped with:

• Intel Core2Duo T6600@2.2Ghz
• 4Gbyte RAM
• Windows7 Home (x64)

The web benchmark suite is loaded from a glassfish AP running on another server, the last GWT 2.5 SDK has been used to compile the benchmarks into Javascript code (Obfuscated javascript).

Let's see the Results

Here you have a bar graph (Figure.1) generated with (The Great) R; the bar heights represents the benchmark execution time, normalized towards the JavaVM 1.7.0 execution time. One little bar means little execution time and high performance, please note that the y-axis has a logaritmic scale.

The experiment name (x-axis) have a number appended on it, bigger the number, bigger is the data size used: FFTBenchmark5 is less memory demanding than FFTBenchmark6.

Take a look at the graph, the first benchmark on the left is the CollectionSortBenchMark3, the red bar represents the JavaVM (ver.1.7.0_05) execution time, the yellow bar is about the Chrome browser (V8 engine) and is about 2 times slower than native javaVM; the other browsers looks about 10 times slower than java code running "native".
The blue bar refers to Microsoft Internet Explorer version 9 (MSIE).

Figure.1
Littler execution times (tiny bars) means higher performance

Last note about these results, I have had to remove a Sieve benchmark on a greater numeric range (big numeric arrays), it's my fault, I have not tuned right the number of iterations and the javaVM has completed the task in ZERO ms!! Quite strange and annoying but I have had not the time to grasp inside this.

Another note, there are few limitations on the browser about the array size, Firefox has an higher limit but the problem applies too. So, sometimes all the JsVMs suck.

Conclusions

Results in Figure.1 are quite self-explanatory.

From a GWT application standpoint Chrome rules, aka V8 rules.
A java program running inside the Chrome browser is going to perform about 2 times slower, but it's so near to JavaVM, so near.

Another interesting point is Chrome regular expression execution time: Java Regex bechmarks performs far better in Chrome than in JavaVM; You know, the GWT Regex classes are backed directly by low level JsVM functions and they look terrific.

Kudos to Chrome regex implementation.

The Java GWT performance in Opera,Firefox and MSIE sucks a bit, running the code on those browsers is 5 or 10 times slower than running on JavaVM.

Firefox sucks a lot with numeric workloads or crunching big arrays of numeric data, running 40 (or 50) times slower than the JavaVM .

I hardly suspect that the GWT developers devote themselves to Chrome optimization, and for this reason GWT compilation for the other browsers sucks. I have no clue, do you have? :-)

From an HPC standpoint, GWT apps on Chrome performs very well compared to native Java Virtual Machine apps, so, GWT looks like a good tool for writing distributed High Performance Computing web clients.

PS:

Comments and criticisms are welcome

  

Linux ReadyBoost alternative aka "how to cache a slow big drive on another quick drive"

I have looked forward a way to cache a big slow drive on another quick drive (in a similar way as Windows7 ReadyBoost already do with flash usb drives).
I have found out this interesting project, flashcache (GitHub) which is a facebook released code;
Flashcache works at the block device level and take advantege of quick SSD drives.
Microsoft ReadBoost caches little files by taking advantage of slower flash device (access time of slower flash device is still lower than hard disk one).
So, ReadyBosst and flashcache are similar but different.
I'll surely try flashcache in the future but it's a shame it doesn't come bundled with vanilla kernels.

More infos here: Facebook FlashCache Post

Self compile Tesseract 3.0.1 on Debian Squeeze, libleptonica problem

In my case, ./configure fails with an error :


checking for asin... yes
checking for leptonica... yes
checking for pixCreate in -llept... no
configure: error: leptonica library missing

Libleptonica package was correctly installed but something is broken; there are tons of workarounds around about setting paths or other magic options, they are useless.
In order to solve the issue you have to uninstall (by apt-get) the debian libleptonica package and self compile libleptonica by source; After libleptonica self installation (make install), tesseract compilation 'll work as expected.
Get leptonica source.
Get Tesseract compialtion instructions.

Openvpn server with Qnap TS-409 Pro, you can

 
Qnap ts-409 Pro is an old but powerful 4-bay NAS box (linux ARM) and comes with a nice proprietary webinterface and lot of functionalities.

The Mission

Add an OpenVPN access to my home network.

I like to have full access to all my home machines (bonus target: grant the home machines access to the VPN subnet as well).

The Plan

Installing openvpn on the main router/firewall is usually a good way to go but my home network has, as usual, a low-end TP-Link ADSL router and there is no way to install openvpn into that device.

The plan is to install the openvpn server instance into my only 24h/24h server, the Qnap TS-409 Pro. 

Warning

Installing a new software, changing configuration files and writing into flash are, for inexperienced users, always dangerous operations.

This guide may apply to other similar Qnap NAS models as well but you have to check for correctenss and applicability of this procedure by your own.

I'm not responsible of any damage, if you brick your box it's your fault, not mine.

The Solution

The Qnap community is very active and the box comes with a lot of QPKG packages, e.g. installing OpenWRT/ipkg support is straightforward.

This is the step by step guide I have used; here is the summary:

1. QPKG Optware installion
2. ipkg update
3. ipkg install of openvpn package
4. manual installation of tun.ko module
5. create CA and certificates (EasyRSA, you can do this your pc)
6. create server configuration files
7. create client configuration files
8. set port forward on router
9. configure new static route on the router
10. enable network forwarding on the TS-409
11. test configuration
12. change autostart.sh in order to persist changes thru reboot

It is mandatory to apply these  OpenVPN_Extras instructions (9,10) in order VPN to communicate properly with LAN; The tun device works only by routing properly the traffic.

In next paragraphs I'm not going to rewrite the linked wiki but I'll focus only on the tricky and interesting parts.

Details

Assuptions and network topolgy

Details about my network topology and choices about openvpn service ports.

Local network IP range: 192.168.55.0/24 (Subnet 255.255.255.0)
Qnap NAS server, home LAN IP: 192.168.55.77

OpenVPN service listens to 192.168.55.77:22111
ADSL Router forwards (Virtual Server) incoming UDP WAN traffic to port 7788 to the home host 192.168.55.77:22111 (yes, different port)

Clients will connect to Openvpn thru an external port which looks different from 22111, my choice

Server configuration

Mount the flash device in order to edit autorun.sh, with command:
mount -t ext2 /dev/mtdblock5 /tmp/createThisDirectoy/

mtdblock5 is the right device name for Qnap TS-409 Pro, check your box guide or wiki in order to find out your own.

... pay attention, after hacking the flash please umount in order to save changes properly:
umount /tmp/createThisDirectoy/

My autorun.sh looks like this, this chages are needed in order to restart openvpn automatically after a system shutdown

mkdir /dev/net;
mknod /dev/net/tun c 10 200;
(sleep 10;insmod /opt/etc/openvpn/modules/tun.ko)&

echo "1" > /proc/sys/net/ipv4/ip_forward

(sleep 15; /opt/sbin/openvpn /opt/etc/openvpn/easy.conf)&

The green row enables ip forwarding inside TS-409, this will route the VPN traffic from the tun interface towards the home lan interfaces, and the box will forward the packets to the destination machines.

my current easy.conf looks like this:

# OpenVPN server configuration QNAP NAS
# basic settings
# you have to choose your port (the listen port of openvpn service)
port 22111
proto udp
dev tun
#
# detect mtu if the connection is slow.
; mtu-test
#
# define mtu, if necessary
; tun-mtu xyz
#
# define the ip-addresses of the underlying tunnel
server 10.8.5.0 255.255.255.0
#
# Route
push "route 192.168.55.0 255.255.255.0"   #  <--- Enter the ip-address of your home network here!
#
# certificates & keys
dh /opt/etc/openvpn/keys/dh2048.pem
ca /opt/etc/openvpn/keys/ca.crt
cert /opt/etc/openvpn/keys/server.crt
key /opt/etc/openvpn/keys/server.key
#
# data compression
comp-lzo
#
# allow, that several clients with the same common name log on
; duplicate-cn
#
# different clients can "see" each other through the tunnel.
; client-to-client
#
# Keepalive
keepalive 15 120
#
# verbosity of status messages in the console. Activate for debugging (1-9) possible)
; verb 5
#
# Log files
status /share/HDA_DATA/.qpkg/Optware/etc/openvpn/log/status.log
log-append /share/HDA_DATA/.qpkg/Optware/etc/openvpn/log/openvpn.log
#
# Run as daemon (activate, after everything is set up properly)
; daemon
#
# Management Interface. Access with "telnet localhost 7505"
management localhost 7505

Notes:
The TS-409 Pro Optware folder is located (by QPKG installer) in a path which differs from the wiki guide, in my case my path is: /share/HDA_DATA/.qpkg/Optware/
Let's check optware path and files location on your box and change the script accordingly.

Client configuration

The clients, with the proper private keys and certificates, connect to the OpenVPN server using this configuration file:

# connect to QNAP OpenVPN Server
# 
port 7788
proto udp
dev tun
tls-client
remote youraccountname.dynds.net 7788  #  <--- enter your dyndns-account here!
pull
# set mtu, if necessary
; tun-mtu xyz
#
resolv-retry infinite
nobind
persist-key
persist-tun
# certificates and keys
# Note the double \\ in the path for a windows config
ca ca.crt
cert your.crt
key your.key

ns-cert-type server
verb 4
comp-lzo


There are redundant settings and useless options too but, you know, this configuration file works :-)


Router configuration

The router has to do two operations:

• forwards (Virtual Server) incoming UDP traffic to port 7788 to 192.168.55.77:22111
• Routes all the packets with destination 10.8.5.0/24 to the gateway 192.168.55.77

The first one is a basic router configuration, usually named "virtual server", the second one is a bit difficult and you net to set a static route into your router.
The rule is this:
  

Destination	Subnet Mask	Gateway	Interface	x
10.8.5.0	255.255.255.0	192.168.55.77	br0

 br0 is the internal LAN bridge.

Why this route?

This static route is necessary for the LAN to send back packets to the VPN.. the LAN's machines are going to send all the VPN packets to the router (the destination is another subnet) and without this rule they are going to be lost o rejected.

Conclusions

I have only changed few details from the original wiki and I have succefully achived my objective.
Being able to connect to my home network thru VPN, being able to connect and communicate to all machines inside.

OO-GC Language for standalone executables (Part 2)

This is an update on the topic.
In the first part I have presented my findings about few languages and tools, hopefully I have inspired a talented friend of mine (I'll refer to him as The Amazing Vlad,or simply The Amazing)
The Amazing works with C#/mono and has tested the Mono's mkbundle for real (in order to create a standalone executable); mkbundle isn't working well as expected, so I have removed mono from the winners list.
Here I paste the translated friend's mail:


Hi,

I am looking torward creating a portable version of KeePass for Linux.

I have installed mono 2.1x on CentOS5 and I have tried creating a bundle for KeePass by using --deps --static.

The first option might bundle all the needed managed assemblies,  the sedcond one might statically link the native mono library. (lib_mono_something...)

It turns out that mkbundle doesn't link the other needed native libraries, e.g. libgdiplus  which is called by WinForms applications.

I have tried using the generated bundle on CentOS6, and I have faced these problems:

 1

It is impossibile to have a priori knowledge (using ldd command) about dynamic library dependencies because managed assemblies load them at runtime .e.g. libc. 

Ldd has stated libc as a dependency but something inside the bundle looked for libc.so which it doesn't exist. The error arises because the real name is libc.so.6.

But how does Mono know about the real name of libc library? Quite simple, mono has a configuration file with a number of mapping directives for windows dll and dynamic linux libraries.

I have fixed the issue by copying and renaming the config as NameExecutable.exe.config  and I have forced mkbundle to add this file to the bundle.

2

The program still has kept failing and ranting about lacking libgdiplus. As mentioned above, no native dynamic library is bundled with mono (by using --static). So, you are forced copying all the needed libraries with your application (plus their own dependencies).

I have copied libgdiplus.so.x and It still failed cause lacking MonoFuckedLongName.dll which doesn't exist in my system!!

I gave up

NOTE: A simple commandline Hello World application still works flawless, with --deps --static ; no tricks or workarounds are needed.

Conclusion

Mkbundle doesn't work well with complex applications and lot of libraries depencies, but it might work well for simple commandline applications. Perhaps WinForms might work by copying libgdiplus with your application but I have not tested a graphical hello world application yet.

 The Amazing