2011-06-27

Some Customizations on the Blog for Displaying and Printing, Vim Helps

I have made some theme customizations and modified two previous posts targeting less JavaScript requirements, shorter load time and some aesthetic.

I save web pages. I always do. I do not bookmark everything, but I print pages in PDF format. This way I can read them later, without making my bookmarks fat. They are already fat enough. I see my readings as my data, and I want to store them. And I go angry against unprintable, very very long pages. I also use ScreenGrab to store the screen-shot of a web page, but it can get stuck with the long web pages.

And I was thinking, my simple blog should be easily printable too. But it was not. Sadly, it seems that Blogger pages(may be not Blogger, but all the themes I have ever used in Blogger) can not be completely printed. Also I am a PasteBin user, and I have used its feature of embedding code pieces to the blog. But that made printing much more difficult. Without PasteBin code embedding, a great site, PrintFriendly, was able to print, but with embedded code from PasteBin, even PrintFriendly could not print the pages. So, I have decided to modify the blog, and make it printable as much as possible.

The first was to remove the embedded code from PasteBin and directly putting the code inside the blog post body. This way I could also control the colors.

As I am a Vim user, it is not difficult to find a cool Vim plugin, ScreenShot.vim to convert code pieces to HTML. By this way, I can easily convert a portion of code to HTML. The output of ScreenShot.vim is different from the built-in "TOhtml" command. ScreenShot.vim provides 3 commands essentially.

Screenshot : Just like it says, it takes the screenshot of the Vim window.
Its output looks like this:

 31 # This is only the file name:
 32 db_name = "our_db.sqlite"
 33 
 34 # The protocol and full path to our database
 35 db = DAL('sqlite://' + db_name, folder=modul

 36 
 37 # Define the table "genre", note that the fi
 38 db.define_table("genre",
 39         Field("name"))

 40 db.genre.name.requires = IS_NOT_EMPTY()
 41 
 42 # Define the table "band", note that the fie
 43 # band.genre_id references to genre.id field
 44 db.define_table("band",

 45         Field("name"), 
raw.py                        40,1           42%


Second command, Text2Html, converts the selected text to HTML. Looks like the built-in TOhtml command, but, TOhtml command creates a page with <html>, <head> etc. On the other hand, Text2Html command creates a portion of an HTML page, starting with a <table> and ending with a </table>, ready to embed in the blog post body or somewhere else. See a modified, previous blog post to have an idea on how it looks like. This is the command I will be using frequently from now on.

And the third command, Diff2Html is used to display two files comparing with differences highlighted.

Also there had been some color inconsistencies in this blog, but finally there will be no more. I will be using Molokai color scheme for Vim, based on Monokai for TextMate, since I like it much and goes very well with the current dark theme of the blog.

The ScreenShot.vim plugin is configurable. Read its script page for more options. Here are my settings from my vimrc targeting saving the maximum screen space:

let g:ScreenShot = { 
    \ 'Title' : 0, 
    \ 'Icon' : 0, 
    \ 'Credits' : 0, 
    \} 

Note that I have deleted the line numbers in the code above using Vim, it has no configuration in ScreenShot.vim it seems, but was a piece of cake using Vim's columnar editing.

Want to see it on PrintFriendly.com? Click here. It will remove the gadgets and other giggling things, and you will see clean, printable version of this blog post. I highly recommend PrintFriendly by the way.

After this modifications, the blog site is faster, more accessible, has more width in the layout, will display anyway if you use a JavaScript blocker like NoScript and it looks better. Have an advice? You are welcome.

2011-06-24

Standalone Usage of web2py's DAL, gluon

For a web project, I had to use the model of the MVC pattern outside the web server. I could do without it, but why to drop it if I could use the model without repeating myself? What I need, is to have a conventional web server serving a conventional web application. So far so good. But I also need to access the database from some other servers which has nothing to do with the web interface.

There are many many web frameworks existing for Python. Among them, both Django and web2py are innovative, great ones. Of course I am not a user of all the frameworks in Python, but these two are the ones that I am focusing on. Both Django and web2py are full-stack web frameworks, that means they provide all the layers of the MVC pattern without using extensions. They are both candidate for the job.

I had some constraints for this project.
  • The system should have an easy install. As I will use it in more than one server, I should not repeat myself in each update.
  • The model layer of the framework must run outside a web server.
  • Database Migration is preferred, but it should be controlled.
  • When I update the model, the others should not migrate, if they do, they would revert the database to the previous state. Only one migration is required. Somehow I must be able to stop them.
  • I should be able to reach my Python libraries installed in site-packages.
Both Django(provides an ORM) and web2py(provides a DAL) seems capable for my task. Django is more explicit, web2py is more automagic. Django supports database migrations using extensions like Django-South, web2py supports migrations by default.

I have attacked the problem using web2py.
Please note that this is not a introductory tutorial for web2py, if it were, would not be better than the official web2py book.

I am using Ubuntu 11.04 Natty amd_64, and the commands will be fully compatible with it, but there is no reason not to work for other operating systems too.

Downloading and installing web2py
This is a great advantage of web2py, it has no configuration nor installation. It works out of the box. I have downloaded the source distribution. And just unzipped it, that's it.

Adding the standalone module
Just created a directory, called "alone" in the web2py directory. This is not a "web2py application", actually it can be anywhere. Added a file "single_stuff.py" inside the directory. Overview of the directory and file structure, is this:


You know what, the DAL of web2py, it can handle all my needs and constraints:
  • The installation is.. There is not even an installation, this constraint is already done.
  • After adding the root folder of web2py to PYTHON_PATH, only some of the modules that are related with gluon can be imported. See how to do it in the source code. Done.
  • Migrations are automatic in web2py but we can fine tune it, we can disabled migrations for some objects. Done, very done.
  • The last constraint, accessing other modules, sure it works, we did nothing unconventional. Run it as "python single_stuff.py" as usual and do whatever you like.
web2py is now the choice for me in this project, for its simple yet powerful database abstraction layer, gluon.

Here is the source of the "single_stuff.py" with inline comments:

Or you can directly jump to see the code on Pastebin.
You can also download whole the code from Pastebin.

"""
An example to use only gluon, which is the name of
web2py's data abstraction layer.
"""

import os
import sys

# web2py is distributed and deployed with zero configuration.
# Let us follow the same approach.
# To do this, I have opened a directory called "alone" inside
# the web2py directory, next to applications.

# I want to distribute the whole web2py directory.
# So, I want to make sure that the gluon is automatically found.

sys.path.append("../")
# print sys.path

from gluon.sql import DAL, Field
from gluon.validators import *

# For simplicity, we will be using SQLite.
# First, determine the path for the database file.
# We are headed for portability, so let us open the database
# file in the same directory with this module.

module_path = os.path.abspath(os.path.dirname(__file__))
# print module_path # => /home/alex/Projects/web2py/alone

# This is only the file name:
db_name = "our_db.sqlite"

# The protocol and full path to our database file.
db = DAL('sqlite://' + db_name, folder=module_path)

# Define the table "genre", note that the field "id" is automatic.
db.define_table("genre",
        Field("name"))
db.genre.name.requires = IS_NOT_EMPTY()

# Define the table "band", note that the field "id" is automatic.
# band.genre_id references to genre.id field.
db.define_table("band",
        Field("name"),
        Field("origin"),
        Field("genre_id", db.genre))
db.band.name.requires = IS_NOT_EMPTY()
db.band.genre_id.requires = IS_IN_DB(db, db.genre.id, '%(name)s')
db.band.genre_id.writable = False
db.band.genre_id.readable = False

# Let us insert some data for table "genre".
# Each insert statement returns the unique_id.
id_rock = db.genre.insert(name="rock")
id_metal = db.genre.insert(name="metal")
id_grunge = db.genre.insert(name="grunge")  # we will use the ID.
id_jazz = db.genre.insert(name="jazz")

# The data will be stored only after a commit.
db.commit()

# We can also do a rollback, before a commit.
print db.genre.insert(name="banana")
# It will print 5 to the screen.
db.rollback()
# But it will not be stored in the database.

# Let us insert some data for table "band".
db.band.insert(name="Nirvana", origin="Aberdeen", genre_id=id_grunge)
# Do not forget to commit.
db.commit()

# A simple select.
for row in db(db.band.name == 'Nirvana').select():
    print row.name, row.origin
    #prints Nirvana Aberdeen

# We can also take the last query.
print db._lastsql
# It will print this:
# SELECT  band.id, band.name, band.origin, band.genre_id FROM band
# WHERE (band.name = 'Nirvana');

# We can use free, raw SQL queries too.
print db.executesql('SELECT * FROM genre;')
# [(1, u'rock'), (2, u'metal'), (3, u'grunge'), (4, u'jazz')] 


Exploring the database
If you want to look into the database using a graphical manager, you can use SQLite Database Browser, which can already be found in Ubuntu repositories, or SQLite Manager, which is a Mozilla Firefox extension and works anywhere.

Here are some screenshots after opening the database created and manipulated by "single_stuff.py":



As a result, web2py is too good to be true, but it is a fact that it is so.

2011-06-19

Using Google Docs to Convert Files

I am a LibreOffice user. That is already too much for me, I can do anything with it. Since I am an Ubuntu user, I do not have to even install that, just I am just adding some of its extensions like PDFImport and LibreOffice base, all from Ubuntu repository. I can open, edit and save all the well known documentation formats. But this is not the case for some people:

Let us say, you are an Microsoft Office 2003 user. And somebody keeps sending you .odt or .docx documents. Some of these people are even using OpenOffice.org/LibreOffice but most probably because of their principles, they are sending you .odt documents which you can not open with Microsoft Office 2003. The people who use Microsoft Office 2007 and keep sending .docx documents to Microsoft Office 2003 users are another story, I will try to keep calm, at least for now.

If you are not able to change or educate the sender, and do not want to install another office productivity suite to your system, there is also another solution : Google Docs.

You can upload the document there, and download in another format you can edit. Here is how to do it:

All you need to use Google Docs is a Gmail account. Login to Google Docs, and upload the document.



You can keep the format or let it to be converted to Google Docs format. Do not worry, it does not hurt, you can do both. Here I have had it converted.


Open the document you just uploaded in Google Docs.


Now you can download the file in a format of your choice using the upper-left File menu, as in the following figure:


Mister must-send-in-docx-so-that-nobody-can-open-my-files-but-i-send-them-anyway Microsoft Office 2007 user, sorry, we can edit your files. May be you think that sending .docx files is a kind of encryption, but it is not. We can open them. Come up with something better.

2011-06-14

A Reference on Yum, Yellowdog Updater, Modified

Recently, I had to mess with yum a lot. It is a powerful command line package management tool for RPM packages, and if you want to do automatic mass installations and updates, as in my case, GUI is not an option. I have collected the commands I have used among others, and created a reference.

All these commands have been tested on Scientific Linux 6.0 x86_64.
The version of yum I have used is 3.2.27.

# A reference to Yum, Yellowdog Updater, Modified
# http://caglartoklu.blogspot.com

# Yum is a command line package manager for
# RPM-compatible Linux operating systems.

Note that these commands requires root privileges.

# _____ Updates

# Check for availability package updates.
yum check-update

# Update all installed packages.
yum update

# Update a specific package.
yum update package_name
# Example:
yum update mc



# _____ Searching and finding about what to install

# List the depencies of a package.
yum deplist package_name
# Example:
yum deplist emacs

# Search a string in the metadata of packages.
# This metadata includes: name, summary,
# description, url.
yum search ri

# Let's say you want the command 'ri' but
# do not know which package is providing it. Type:
yum provides ri
# and you will see that 'ruby-ri-1.8.7.299....'
# package os providing it.

# Information about a package.
# Note that the package does not have to be
# installed on your system, it will display
# the information anyway.
yum info package_name
# Example:
yum info mc

# Install a package or packages.
yum install package_name
yum install package_name1 package_name2
# Example: yum install mc

# List the packages.
yum list all
yum list available
yum list updates
yum list installed
yum list extra
yum list obsoletes
yum list recent



# _____ Installing and removing packages

# Remove a package or packages from your system.
yum remove package_name
yum remove package_name1 package_name2
# erase command can also be used.
yum erase package_name
yum erase package_name1 package_name2

# Install a package or packages.
yum install package_name1 package_name2
# Example:
yum install mc emacs

# You can also install a package for a specific
# architecture like this:
yum install package_name.architecture
# See 'Specifying package names' in this document.
# Architecture can be x86_64 or i386.
# Example:
yum install mc.x86_64

# Install a local RPM.
yum localinstall package_name

# Update the distro using the RPMs saved locally.
# cd to the directory you saved your rpms, and type:
yum localupdate *.rpm



# _____ Downloading packages

# Download a package (current directory by default)
yumdownloader package_name
# If you do not have the yumdownloader,
# install yum-utils using:
yum install yum-utils



# _____ Groups

# See the list of groups installed:
yum grouplist
# This command will first list the installed groups,
# then the available ones.

# After seeing the list, see the information about.
# about a group. If the group name includes a space,
# enclose the group name with ' character.
yum groupinfo group_name
# Example:
yum groupinfo 'FTP server'

# Install packages of a group. Note that default
# and mandatory ones will be installed.
yum groupinstall group_name
# Example:
yum groupinstall 'FTP server'

# Update packages of a group.
yum groupupdate group_name
# Example:
yum groupupdate 'FTP server'

# Remove all packages of a group.
yum groupremove group_name
# Example:
yum groupremove 'FTP server'



# _____ Specifying package names

# A package can be referred to for install, update,
# list, remove etc with any of the following:
name
name.arch
name-ver
name-ver-rel
name-ver-rel.arch
name-epoch:ver-rel.arch
epoch:name-ver-rel.arch


# _____ Shell

# To start the interactive shell, type:
yum shell


# _____ References

http://www.linuxcommand.org/man_pages/yum8.html
http://yum.baseurl.org/
http://en.wikipedia.org/wiki/Yellowdog_Updater,_Modified 


If you do not see the code, you may be running a JavaScript blocker. Then you can directly jump to the source of the code on Pastebin: http://pastebin.com/dbCHYEh0

Enabling Caching Yum Downloads on Linux

I needed to copy the packages and updates performed on a Scientific Linux 6.0 x86_64 (call it A, connected to Internet) to another server(call it B). This second server has no Internet connection, and can not access to repositories but it is the main server in a project. So, I am keeping the server A as "mirror" of the server B. By this way, I can copy the updates as RPM files from server A to the unconnected server B.

Here is what I did to perform it using Yum:
First, make sure that the downloaded packages are cached in server A, that is, they are not deleted after the installation. To do that become root and type

vi /etc/yum.conf

And edit the line starting with keepcache, make sure that it is like this:

keepcache=1

Note your cachedir there.
It is defined as

cachedir=/var/cache/yum/$basearch/$releasever

It is expanded in my system as:

/var/cache/yum/x86_64/6.0/sl/packages

Now, whenever you use yum from command line or GUI, the downloaded packages can be found there and copied freely.