The Power of C in the Cloud

In my last post, I introduced you to the cloud IDE Codenvy. The primarily reason I used it is because it has a simply way to hook into Google App Engine. However, for a more robust cloud IDE, I would recommended Compilr. Compilr has support for many of the languages Codenvy does (Ruby, Java, PHP, Python) but it also has support for Objective-C and C# as well as low level languages like C and C++.

That last point leads to some interesting results. Because we have access to such low level languages, we can use the Compilr servers for general computing. From what I can tell, the Compilr servers that execute the code are running a Unix-like operating system on x86 hardware. So if you’re in a pinch and need access to a Unix like environment, Compilr can be very useful. I happened to run into such a situation while trying to make use of WoLF PSORT for a rather long sequence and lacking the necessary Linux PC to compile and run the program locally.

So, how do you execute commands on commands on Compilr’s servers? First, create your C++ project. Your program will be run in the “content” folder of your project so if you have any external files you need, place them there:

 

If you get an error message when uploading, but you also get “Upload complete”, you upload has finished. Refresh the page for the files to show up in the project panel.

The key to our program is the system function. Simply hand it commands to execute and it will do so. To untar our WoLF PSORT file execute the following:

system ("tar -zxf WoLFPSORT_package_v0.2.tar.gz");

Remember to mark the files for execution if nessessary:

system ("chmod 777 ./WoLFPSORT_package_v0.2/bin/psortModifiedForWolfFiles/psortModifiedForWoLF");

There are a few restrictions Compilr has placed on free accounts.

First, you are forced to make public project and you are restricted to 50 builds/runs. These aren’t huge problems and you can always create a new account and port code if you go over 50 builds. In fact I have noticed the build counter often reset after refreshing the page, so this never became an issue for me.

Second, you are limited to 4000 characters of output. A very simple solution, pipe output into a log file by appending > log.txt to your command:

system ("./WoLFPSORT_package_v0.2/bin/psortModifiedForWolfFiles/psortModifiedForWoLF -t ./WoLFPSORT_package_v0.2/bin/psortModifiedForWolfFiles/all.seq > log.txt");

You may also need to pipe errors and warning, in which case you can log to the same file by appending >log.txt 2>&1 to your command. Or you may wish to disable errors and warnings completely while still capturing output, in which you would append the following 1>log.txt 2>/dev/null .

Third, if your program has not accepted input in 40 seconds, it will be terminated. This requires a tricky work around. I ended up spawning a child process to accept input while the parent process was busy. The child process will send an alert for input every 30 seconds. If you choose to do something similar, remember to flush stdout before sleeping. The syntax is as follows:

fflush(stdout);

sleep(30);

Interestingly, this method triggers a security measure when one process exits before the other and will terminate the program. So make sure no process exits before your program has finished doing it’s work.

Some final points about Compilr:

Much like Codenvy, Compilr has issues with Internet Explorer. I managed to get it working in Firefox.

The first compile of every session will fail with “Lost connection to server”. Simply use a dummy application to trigger it and continue your work. If you run an application that takes a long time and it loses connection, it could block you from running further applications until it finishes.

Always backup your progress! I can’t stress this enough. If the server that stores the code loses connection to the server that runs the code while “Syncing application data”, you can lose large chunks of your work.

Finally, I’ve attached some sample code on the download website.

Programming with Java on Google App Engine

I wrote a draft post about Google App Engine many moons ago when it was Python only. As I’m not a Python man, the post isn’t very technical. Anyway, things have changed and GAE now supports Java. Albeit a customized subset of Java, which I’ll go into later.

Google App Engine is a cloud computing service: run your programs on their servers and pay by CPU usage. It’s similar to Amazon’s EC2 and Microsoft’s Azure, but unlike them in there is an absolutely free trial. You’re only required to provide a telephone number when signing up.

Step 1: Create the proper account

Sign up for a Google account here and a Google App Engine account here. After the GAE account is created, you should be prompted to create a new Google App Engine ID.

Now, if you’re like me, I won’t want to have to download and install the various SDKs and IDE to make your GAE app. So, I suggest Codenvy. Be sure your logged in to your Google account and click “Connect with Google”. Be sure you’re not using Internet Explorer, Codenvy seems to have issues with it.

Step 2: Create a project

On the welcome screen, choose “Create a New Project From Scratch”

Choose  Java Web Application (WAR) -> Google App Engine

You have to select the sample project, but you can skip using the JRebel plugin.

Use the Google App Engine ID you just created.

Step 3: Clear out the sample code

Click the project tab in the left menu (next to the Package Explorer tab)

Delete the src\main\java\com folder.

Delete the src\main\webapp\display.jsp file.

Open src\main\webapp\WEB-INF\web.xml and empty the “servlet” and “servlet-mapping” items. You can add java code to the src\main\java folder at a later time and add the proper information in these tags to specify it as a servlet.

Add the following in the “web-app” item after the “servlet-mapping” item:

<welcome-file-list>

        <welcome-file>index.jsp</welcome-file>

 </welcome-file-list>

Open src\main\webapp\WEB-INF\appengine-web.xml and make sure the “application” has enclosed your Google App Engine ID

 Customize src\main\webapp\index.jsp as you wish.

Step 4: Run your app on Codenvy servers

Make sure you save! Codenvy does not auto save changes when building.

Run -> Run Application

Step 5: Load your app into Google App Engine:

Make sure you save! Again, Codenvy does not auto save changes when building.

Go to Project -> PaaS -> Google App Engine

Select Application Update

If you get an Oath token error:

PaaS -> Google App Engine -> Logout

PaaS -> Google App Engine -> Login

Also, see this link for setting up cron jobs.

Web development: HTML, PHP, and SQL

First thing’s first, lets set up our environment. My IDE of choice is Netbeans, but that’s of little consequence. Next, install your PHP stuff. Later down the road, when we use MySQL, these tools will make yourlife a lot easier. In your php.ini file uncomment - extension=php_curl.dll to activate curl.

This post will tell you how to set upyour project. The easiest thing to start with is how to scrape a feed, basically cut and paste, see here. Quick posts to brush on your PHP:

http://php.about.com/od/advancedphp/ss/php_preg_3.htm

http://php.net/manual/en/language.types.array.php

http://www.w3schools.com/php/php_if_else.asp

http://www.tizag.com/phpT/postget.php

http://php.net/manual/en/function.var-dump.php

Now let’s tap into Google APIs. Sign up for your Google License Key. Read how to use the Google API here and here. The Google News API is similar.

Next API were going to use is Twitter, well technically 3 Twitter APIs. The first is to get trending topics, seehere. Next, twitter searches, similar to Google’s search API, but you don’t need a key, see here. The third is a bit trickier, see here for help. Basically download Abraham's TwitterOAuth library add it to your project and go to http://dev.twitter.com/apps, register your app and get your keys.

Now before we jump in and start using SQL, I feel it’s important that you keep in mind the scope of your project. If you simply want to store a small amount of information with no respect to order, you might consider plain text. There’s a certain cost to setting up and maintaining a database and it may even hurt performance if you are just doing a simple project.

OK, ready? Start your server. Open up MySQL Query Browser. Right click on a Schema, create a new one or use a default, and create a new table. Now, let’s connect our new table to our PHP, see here. We’ve been getting json objects for our searches, so let’s store those in the database. For help see here, here, and here. I managed to get away with just using serialize() and mysql_real_escape_string() but not addslashes().

And there you have it. You are now the proud creator of a web application.

Photosynthesis and Cellular Respiration

I was going to make a detailed post on the formation of kidneys (mostly to make the joke “Zac Efron the Nephron”) but I found the subject to dry and uninteresting. However photosynthesis and cellular respiration is a much more interesting subject in my opinion. In a way it is the basis of all life.

Photosynthesis is the process by which plants (and certain other organisms) store solar energy in carbohydrate molecules: 6CO₂ + 6H₂O + photons => C₆H₁₂O₆ + 6O₂. This is how most new energy enters the food chain. This reaction takes place in the chloroplast. Photosynthesis is split into light dependent and light independent reactions.

That internal structure is called the thylakoid. The light dependent reaction takes place right at the membrane of the thylakoid. Basically, this reaction takes sunlight and uses it to attach a phosphate to ADP to create ATP (the energy “currancy” of the cell) and hydrogen to NADP+. The molecules hold the energy in these bonds. For a summary of the light dependent reaction, see below.

The light independent reaction (aka the Calvin Cycle) occurs outside the thylakoid in the lumen of the chloroplast. It basically uses the ATP and NADPH created in the light dependent reaction to take CO₂ and create carbohydrates.

Cellular respiration is kind of the opposite of photosynthesis. Carbohydrates break down and release energy: C₆H₁₂O₆ + 6O₂ => 6CO₂ + 6H₂O + energy. The main method of cellular respiration is aerobic respiration. A lot of biological reactions operate as a cascade of small reactions, and aerobic respiration is just such a reaction. See below for a summary:

Glycolysis is a 2 phase reaction takes glucose and creates 2 pyruvates.

The pyruvates are then converted to Acetyl coenzyme A.

The citric acid cycle converts Acetyl coenzyme A to coenzyme A. This creates 2 ATP.

Electron transport generates a whopping 32 to 34 ATP!

Anaerobic respiration (fermentation) requires less oxygen, but releases less energy. Glycolysis runs and releases 2 ATP, but pyruvate is converted to lactic acid or alcohol depending on the organism instead of continuing on through respiration.

Now, you’re asking, what about all the other stuff we eat, how do we get energy from that. Well they enter at different parts of the pathway. See below.

Creating a DLL in Visual Studio

This was another subject neglected in my formal schooling. I picked it up through trial and error, but you won’t have to as I’m about to run through an example for you.

In this example I’ll be using Visual Studio 2010 to create a dynamic link library out of a C file and use it in a managed C++ project.

For the C DLL:

Create a Win32 Project. Application type: DLL. Empty Project.

Create your C file. I called mine retnum.c.

The text of my retnum.c is very simple:

__declspec(dllexport) int retnum()

{

        return 5;

}

The __declspec(dllexport) lets the compiler know we want to use this function in a DLL.

For the C++ project.

Lets create a simple CLR Console Application project under the same solution as our C project.

Right click on your solution and go to properties, set startup project to be your C++ project. Also, go to project dependencies, on the “Project” dropdown menu select you C++ project and check off your C project.

Right click on your C++ project and go to properties. Go to Common Properties, Frameworks and References, click “add new reference”. Select your C project.

Create a header file under your C++ project. I called mine cdll.h.

The text of my cdll.h is:

extern "C"

{

   retnum();

}

The extern "C" lets our C++ program know we will be using C code.

The meat of my C++ project was in my app.cpp file. Condense as follows:

#include "stdafx.h"

#include <iostream>

#include "cdll.h"

using namespace System;

using namespace std;

int main(array<System::String ^> ^args)

{

    int i = retnum();

      Console::WriteLine(L"Your number is: "+i);

      cin.get();

    return 0;

}

So, that’s how you create a C DLL and use it in a C++ project in Visual Studio.

Project files now available on the download website.

Compiling on the Wii: a Lesson in Makefile

Makefiles are files used to make compiling easier on Unix like systems.

Now I know what you’re saying. You’re saying, “Centraldogma, what does the Wii have to do with Make files?”. Well, because I learned it through trying to make some Wii homebrews, so that’s how I’m teaching it. Sure I learned some basics in college, but not the level you’ll see in Open Source.

But, let’s start with the basics. Here is a basic makefile:

all: hello

hello: main.o factorial.o hello.o

        gcc main.o hello.o -o hello

main.o: main.c

        gcc -c main.c

hello.o: hello.c

        gcc -c hello.c

clean:

        rm -rf *o hello

Each has word with a colon after is a rule. all is a default rule, if you create a file called makefile and say “make” the all rule will be executed.

Immediately after the colon is the requirement for the rule to be used, if these are other rules, those other rules will be executed. In this case all requires hello. hello requires main.o and factorial.o. For main.o a file called main.c must exist in the current directory and must be compileable. The same for hello.o and hello.c.

So, all you have to do is have your main.c and hello.c with your make file and type “make” and you get the hello binary. If you want to just make an intermediary, say main.o, you would say “make main.o”.

But lets say your all done, but you want to tar everything up and send it to your friend/partnet/employer/anyone-who-will-look-at-your-code. You have all these nasty binaries and intermediaries all over your directory! Well, we have this other rule, clean, which removes any file that ends with .o and the hello binary. Just say “make clean” and it’s done.

Well, that’s the basics, on to the advanced stuff. That’s where the guys at CodeMii come in, I’ll let them explain it:

A makefile is usually found that the root directory of the source code you have. Take the gamecube template source code example (C:\devkitPro\examples\gamecube\template). In that directory you will see a file name “Makefile”. If you open it up with notepad, you’ll be able to see the different parts of a makefile.
The important parts of a makefile for this tutorial are the following lines:

1.include $(DEVKITPPC)/gamecube_rules

By specificing “gamecube_rules” we are telling our compiler that the source code we want to compile will be run on the gamecube. You would use “wii_rules” to compile for the Wii.
By changing between these two, you are changing which libraries will be used when compiling the source code. Libraries are a bunch of files which we use in our source code to interact with the gamecube or Wii system.

1.SOURCES        :=    source

2.DATA        :=    data

3.INCLUDES    :=

These lines tell the compiler which files should be compiled. Most of the time you can leave this alone as our source code will be in /source and our additional files like images, music, etc will be in /data.

1.LIBS    :=    -logc –lm

This is a very important line as it tells the compiler which additional libraries we wish to use. Say we want to play an mp3 and have the relevant code in our source to do so. If we were to compile our source with the above line, the compiler would complain and say that it can’t find the functions we are using to play an mp3 file. For playing mp3 files, you need to include the “lmad” library.
The LIBS line when we add the lmad library looks like:

1.LIBS    :=    -lmad -logc –lm

The order of how you include your libraries is also important as some libraries may reference other libraries and if you haven’t got them in the right order, the compiler will complain about it.

SSH and Linux

SSH and Linux

Meant to put this up yesterday, but better late than never.

I guess this is my first technical post about programming. Well, not really programming… whatever.

My first experience with Linux went something like this: install whatever the popular distro of the day was, get any additional hardware working right, play around with it for one or two day, then format and install Windows. Because OS that you use really isn’t all that important as long as your familiar with it.

When I went off to college, I got much more rigorous training in Linux (and all Unix based systems) trying to compile code on the lab machines through CLI.  Was like a whole different experience. And I am going to impart some of the wisdom I gained to you my loyal reader (lucky you!).

Something important about Linux: Everything in Linux is a file. Also, see the picture for the breakdown of the directories.

If the place you’re working at supports SSH, your weapons of choice will be PuTTY (for general CLI) and Webdrive (for SFTP file transfer). PuTTY is free but Webdrive will cost you money. There are free software that does Webdrive’s job, but in my opinion, not as well. PuTTY can also be used for tunneling, which can be useful in … circumstances.

ls = list files in a in a directory. Use “ls –a” for detailed info. In case you don’t know –a is an argument to the program ls, you can pass multiple augments to a program as such “ls –a –rf” ect.

pwd = gives your current directory

cd = change your current directory. “cd ..” will move you to your parent directory as “..” represents to the parent directory

. = at is a way to denote the current directory. Say you need your current directory to run your program “a.out”, you’re going to want to say “./a.out”

mkdir = create a directory. To create a directory called foo say “mkdir foo”. You can also use –m to specify a mode.

rm = remove a file. You can’t use rm to remove a directory unless you use “rm -rf” to recursively go into each folder, remove it’s condense and remove the folder.

more = print out the contense of a file

emacs = emacs is a popular text editor. Hit Ctrl-X and Ctrl-C to close emacs.

top = displays all the processes running. Like a task manager but CLI.

gcc = C++ comiler. –o tell it what to name the compiled file. –Wall gives you more warnings, but those are just warnings, it won’t break your program. –pedantic –ansi holds the program to stricter standards. –g compiles for debugging with gdb (I never used gdb, though it has some interesting features, I stuck to printf’s and getf’s to debug when I couldn’t use a proper IDE).  So the whole thing would look like “gcc-o hello hello.c –Wall –pedantic –ansi –g”

tar = –cvf creates the archive as such: “tar -cvf file.tar file1 file2 file3” tars file1, file2, and file3 into file.tar.  –xvf untars a tar file.

scp and sftp – in certain cases, you won’t be able to use Webdrive or your SFTP client of choice. Use scp to send files, sftp to get.