Pragmatic Development Notes

2008-12-17

With A Little Help From Ruby

Filed under: C++,General,Ruby & RubyOnRails — Boško Ivanišević @ 9:20
Tags: , , , , , , , ,

Recently I had to work with Outlook and its OLE objects. More precisely I had to handle drag and drop of Outlook folders into an ActiveX control made in C++. Anyone who has ever worked with OLE in C++ knows that it is anything else but fun. Debugging support in Visual Studio is quite poor. Since it is impossible to see even values of properties and Visual Studio will not allow you to call OLE methods neither in Watch nor in Quick Watch windows you are forced to trace messages and that enlarges code one have to type.

It is needlessness to say that Microsoft’s documentation about Otlook’s folders drag and drop was of no help. The most I could find is that Outlook can handle this and that is the way how you can send someone a link to your Inbox on the Exchange server. Not so useful isn’t it?

Examining available clipboard formats that Outlook sets during folder drag, I’ve found out that I can get a file with the .xnk extension. Unfortunately those are binary files with no explanation of their format. One more problem on the list to be solved, but at least something to start with. Quick look at the file in hexl-mode in Emacs revealed that 20 of last 24 bytes of the file keep something that looks like ID.

Hexadecimal view of Outlook's .xnk file

Hexadecimal view of Outlook's .xnk file

But I had to check that. It was time to start up Visual Studio and make a new project where I’ll be able to test it, right? Well, not exactly. Let’s see what would procedure for testing ideas look like in C++. First I would have to make OLE wrapper classes around Outlook’s objects. I’ve tried that several times before and for some reason Visual Studio didn’t add all methods and properties to wrapper classes. But if we assume that main methods would be implemented (which, naturally, turns out to not to be true) we would have something to start with. Since it is unknown which classes we would need we’ll have to make wrappers incrementally.

Next step would be to extract ID from .xnk file and to try to get Outlook folder. And at that early stage we are entering unknown. Should we use Folders collection and iterate over each member of collection or there is some shortcut? Searching through documentation will give us an answer – we can use Outlook’s Session GetFolderFromID method. But there is method with the same name in Namespace object too. Which one should we use and do both of them return same object? Obviously we would have to try both.

That means creating wrapper classes for Namespace and Session objects, call their GetFolderFromID methods and trace some properties of the resulting object. And if we missed, we would have to go through implement-build-debug procedure again. Moreover there is a big chance that some of the methods we need will be missing in wrapper classes. In that case we will have to find method’s dispatch ID, its parameters and to implement method call through InvokeHelper (there are other ways too, but it is not relevant to this article) manually.

Obviously further experimenting in C++ would be time consuming and I needed something faster. It was time to fire up Emacs and to experiment with Ruby. I’ve created class which I’ll use to work with Outlook’s OLE interfaces.

require 'win32ole'

class Outlook

  def initialize
    @outlook_app = WIN32OLE.new('Outlook.Application')
    @mapi_namespace = @outlook_app.GetNameSpace('MAPI')
  end

end

Initialization method will create necessary members. First one to work with Outlook application and the second one is MAPI Namespace object which is needed to get and create messages, folders and other Outlook’s objects.

Next step was to load .xnk file, extract ID and to try to get Outlook folder for it.

  # Returns Outlook item defined by ID passed in
  # the argument during function call.
  def get_item(item_id)
    begin
      item = @mapi_namespace.GetItemFromID(item_id)
    rescue WIN32OLERuntimeError
      item = nil
    end

    return item unless item.nil?

    begin
      item = @outlook_app.Session.GetFolderFromID(item_id)
    rescue WIN32OLERuntimeError
      item = nil
    end

    item
  end

  # Loads .xnk file (got during drag n' drop of Outlook
  # folder) and extracts ID of the folder.
  def get_folder_from_xnk(path_to_xnk)
    puts("Processing #{path_to_xnk}...")
    xnk_size = File.size(path_to_xnk)
    puts("File size is #{xnk_size}")
    xnk_file = File.new(path_to_xnk)
    # Read last 28 characters
    xnk_file.pos = xnk_size - 28
    all_xnk = xnk_file.read(28).unpack('H*')
    # Entry ID is represented by first 48 array members
    item_id = all_xnk[0][0..47]
    puts("Folder id: #{item_id}")
    folder = get_item(item_id)
  end

First method creates either Outlook item or Outlook folder. Since class will not handle only folders with ID given in .xnk file, but also messages in those folders we need one method which we can use to extract them. Second method reads .xnk file, extracts folder ID and returns Outlook’s folder.

Now when we have folder it is not hard to extract items stored in it. Here is complete class:

# This is utility class for manipulating and
# examining MS Outlook content.
#
# Author::    Bosko Ivanisevic (bosko.ivanisevic at gmail.com)

require 'win32ole'

class Outlook

  OL_FOLDER = 2

  def initialize
    @outlook_app = WIN32OLE.new('Outlook.Application')
    @mapi_namespace = @outlook_app.GetNameSpace('MAPI')
  end

  # Returns array of messages found in the folder.
  def get_folder_messages(folder)
    messages = Array.new

    begin
      return messages unless OL_FOLDER == folder.Class

      table = folder.GetTable
      while (!table.EndOfTable)
        row = table.GetNextRow
        message_id = row.GetValues[0]
        msg = @mapi_namespace.GetItemFromID(message_id)
        messages << msg unless msg.nil?
      end
    rescue
      puts "Error! Argument is not Outlook folder"
    end

    messages
  end

  # Method returns Hash of folder content. Folder name is
  # stored under :name key. All sub-folders are stored in
  # the array under :folders key, and all messages are in
  # the array under :messages key.
  def get_folder_content(folder)
    content = Hash.new
    return content if ( folder.nil? || folder.Class != OL_FOLDER)
    content[:name] = folder.Name

    content[:folders] = Array.new
    1.upto folder.Folders.Count do |index|
      sub_folder_content = get_folder_content(folder.Folders(index))
      content[:folders] << sub_folder_content
    end

    content[:messages] = get_folder_messages(folder)

    content
  end

  # Regurns Outlook item defined by ID passed in
  # the argument during function call.
  def get_item(item_id)
    begin
      item = @mapi_namespace.GetItemFromID(item_id)
    rescue WIN32OLERuntimeError
      item = nil
    end

    return item unless item.nil?

    begin
      item = @outlook_app.Session.GetFolderFromID(item_id)
    rescue WIN32OLERuntimeError
      item = nil
    end

    item
  end

  # Loads .xnk file (got during drag n' drop of Outlook
  # folder) and extracts ID of the folder.
  def get_folder_from_xnk(path_to_xnk)
    puts("Processing #{path_to_xnk}...")
    xnk_size = File.size(path_to_xnk)
    puts("File size is #{xnk_size}")
    xnk_file = File.new(path_to_xnk)
    # Read last 28 characters
    xnk_file.pos = xnk_size - 28
    all_xnk = xnk_file.read(28).unpack('H*')
    # Entry ID is represented by first 48 bytes
    item_id = all_xnk[0][0..47]
    puts("Folder id: #{item_id}")
    folder = get_item(item_id)
  end
end

This simple yet powerful class can be used for lot of Outlook content handling. Although it uses only small number of OLE interfaces and is made for specific type of problem it can be easily extended to perform even more complex tasks. Naturally if you want to see what you can do with Outlook automation you should visit MSDN Outlook Object Model Reference.

Bottom line is that using Ruby (or some other scripting language) can really speed up a lot development of applications that use OLE objects. Experimenting with Outlook automation in C++ would be very time consuming. Just imagine all those implement-build-debug circles needed to find out that you used wrong OLE object. And even MSDN documentation looks quite detailed, once you dig into the problem you realize that lot of things are missing or poorly explained. Even more, most of their samples are written in VB and very few mention C++.

Just to avoid misunderstanding, I do not want to diminish power of C++ here. Personally I think that it is still one of the most powerful programming languages. At the end, my final implementation was in C++. The whole idea was to give a new perspective on the process of development which can be accelerated using some helper tools.

This is not the end of story. Ruby gives you a lot more possibilities for exploring OLE objects. Just to mention few:

# Returns array with names of all OLE methods...
my_ole_object.ole_methods
# ... or get properties...
my_ole_object.ole_get_methods
# ... and, of course, put properties
my_ole_object.old_put_methods

Further you can get Ruby’s WIN32OLE_METHOD from the object and explore its properties:

# Get WIN32OLE_METHOD
method = my_ole_object.ole_method(method_name)

# Now you can get its dispatch ID that is needed
# if you want to call it from C++ through InvokeHelper
disp_id = method.dispid

# Or get methods return_type...
ret_type = method.return_type

# ... number of parameters
param_no = method.size_params

# ... number of optional parameters
opt_param_no = method.size_opt_params

For further details you can look at Ruby’s win32ole library documentation.

For my mind it is good way to go. Using Ruby for exploring OLE object you are not familiar with will result in fully functional prototype much faster than if we try to do the same in C++. Once we have prototype in Ruby turning it in final C++ code is quite straightforward and much easier.

Although complete article is about Outlook, similar approach can be used with any of Office applications: Word, Excel or PowerPoint. It also shows that Ruby can be not only helper, but also very powerful Office automation tool.

Advertisements

2008-11-27

Improved Emacs Customization

Filed under: Emacs,Ruby & RubyOnRails — Boško Ivanišević @ 14:44
Tags: , , , , ,

While I was writing first three articles about Emacs:

with somewhat clumsy titles because they are related to Linux too, I was changing my .emacs file. (Due to the WordPress restrictions I had to rename it. If you want to download it just right click on the link and save it somewhere under .emacs name. It is pure text file although it has .doc extension.)

I’ll briefly describe all changes I’ve made. First I’ve defined variable my-root. It points to the root folder where directory .emacs.d is. I needed this just because of frequent switches between Ubuntu and Vista. This change makes it easier to update Emacs configuration no matter whether it was changed on Ubuntu and copied to Windows or opposite. I only need to set my-root to path according to the system I work on.

I’ve also started Emacs server. With it only one instance of Emacs can be used, both on Linux or Windows, when various files are edited. If you want to be able to use it from Windows shell you’ll need Emacs Shell Extension Registry File. Right click on the link, save it with .reg extension and double click on saved file. It will add ‘Open in Emacs’ item in file pop-up menu on Windows.

Next change is related to font setting. It is well documented in .emacs file so I’ll not write about it here. Just will point out that if you want to find XFLD name of the font on Windows you’ll have to switch to *scratch* buffer and type:

(w32-select-font nil t)

After that just press C-j (Ctrl-j) at the end of the buffer and after you select font you want you’ll get the name you should use in your .emacs file.

I’ve also set default tab width to 2, enabled ANSI colors in the shell and set mode for editing .ini files.

Near the end of .emacs file there are several lines related to Cygwin. If you want to use Windows (GNU) version of Emacs under Cygwin you’ll have to uncomment these lines and to download cygwin-mount.el package.

Final change is something I really like, so I’ll describe it in more details. If you read previous articles you already know that I really enjoy working on Ruby and Ruby on Rails projects in Emacs. There was only one thing I was missing – navigation within the file. In Rails mode pressing M-S-down (Alt-Shift-down arrow) pops up menu for quick switch to files related to current context. For example if you do that while you have controller file opened you’ll get pop up menu with items to jump to Functional Test, Helper or views.

Pop up menu in Rails mode.

Pop up menu in Rails mode

What I needed is similar functionality but for navigation within the file which will make it easier to jump to function definition. Under IMENU menu item you can find all function definitions and you can switch to each of them but you must use mouse. There is one more way to do it with the mouse C-mouse 3 (pressing right mouse button while keeping Ctrl button pressed). (BTW two other useful shortcuts are C-mouse 1 (Ctrl and left mouse click) which opens buffers menu and C-mouse 2 (Ctrl and middle mouse click) which opens text properties menu.) Lets get back to navigation problem. Obviously I could use mouse to navigate through a file but I didn’t like it. I prefer to have keyboard shortcut for that. On the Web I’ve found are few examples related to Emacs pop-up menu and keyboard handling, and that was my starting point. I’ve modified these examples and made small menu-util.el package loaded at the end of .emacs file.

This package adds few things I really like. With C-M-down (Ctrl-Alt-down arrow) it will display major mode menu (which is the list of functions in Ruby file, for example).

Easier navigation in Emacs.

Easier navigation in Emacs

C-M-up (Ctrl-Alt-up arrow) pops up main menu so you will not need mouse any more.

Main menu as a pop up.

Main menu as pop up

Finally M-up will open completion pop-up menu while you are working on your code.

Completion menu in Emacs

Completion menu in Emacs

Since I really do not know Lisp I’m sure all these could be made much better. If any Lisp guru has an idea how to do that I’ll be very happy to hear suggestions. Until then I can live with what I’ve made and I hope you’ll find it useful too.

2008-11-05

Emacs On Windows – First Part

Filed under: Emacs,Ruby & RubyOnRails — Boško Ivanišević @ 2:23
Tags: , , , , ,

This is first post in small series of articles about Emacs. Although its title indicates that it is only about Windows most of it can be applied on Linux. Actually there are just a few Windows specific things in it. The goal is to get excellent environment for Ruby, Ruby on Rails development and tool for writing posts.

In order to set up Emacs you, naturally, have to download it first. You can download Xemacs, EmacsW32 or GNU Emacs. My favorite is GNU Emacs. There are several reasons for it. First it doesn’t come with the installer so I can always completely remove it if I don’t like it, without any fear that some files will be left as is so usual for the most of the applications on Windows. Second, on GNU Emacs download site I can always download latest version as soon as it is released. I do not have to wait patches to be applied or some installers to be made. Simple and clean.

Installing Emacs on Linux depends on the flavor you use. Ubuntu is the one I like the best. Installing Emacs on Ubuntu is quite easy. If you want official release just use:

sudo apt-get install emacs22

On the other hand if you want to see what new version will look like do:

sudo apt-get install emacs-snapshot

By the way I’m writing this post in snapshot version (23.0.60.1) on Ubuntu 8.10.

I like to keep all good and useful things grouped, so I keep them in C:\Utils folder. After downloading zip archive I’ve unpacked Emacs in C:\Utils\emacs-22.3 folder. Of course, you can unpack it wherever you want. Anyway all customizations and additional packages will be in separate folder.

One small digression. I use Cygwin a lot on Windows. Unfortunately current official version of Emacs on Cygwin is 21.3. It is quite old version and some of Lisp add-ins I use do not work with it. Version 22.1.3 which is marked as experimental always hangs both on Vista and XP and I could not use it. So I’ve decided to use same version on Cygwin. Some of the customizations will be Cygwin specific but I’ll make a note so you can omit if you don’t need them.

Now when Emacs is up and running it is time to customize it. First little introduction. During start up Emacs reads .emacs file from home folder. On Linux everybody knows what home folder is (/home/[user_name]), but on Windows home folder is different if you are on Vista (it has everything different, unfortunately) or on some earlier version. On Vista, home folder is in C:\Users\[user_name]\AppData\Roaming, and on XP it is in C:\Documents And Settings\[user_name]\Application Data. If you don’t like where Emacs searches for its start up file you can always change it by setting HOME environment variable to something else what is more suitable for you. I do not do that because I try to keep things as they are by default because it is easier for me to restore complete system if I need it.

Finally here is the first part of my .emacs file:


; Library with common Lisp extensions for Emacs.
(require 'cl)

; Needed to see how fast Emacs loads.
; Loading time is printed at the and of the
; execution of .emacs file.
(defvar *emacs-load-start* (current-time))

; I really like this font. I also tried Monaco which you can
; see on lot of Railscasts but I couldn't find the one which
; supports Serbian Cyrillic and Latin letters.
(set-default-font "Bitstream Vera Sans Mono-9")

; All customizations will be saved in separate file. This way
; it is easy to remove customizations and keep this file clean.
(setq custom-file "~/.emacs-custom.el")

; Now load all customized values.
(load custom-file 'noerror)

; This should allegedly speed up Emacs starting by preventing
; some requests from the window manager back to the Emacs. Frankly
; speaking I didn't notice some speed up but I still keep it:(
(modify-frame-parameters nil '((wait-for-wm . nil)))

;Allows syntax highlighting to work, among other things
(global-font-lock-mode 1)

; Sets initial window position
(set-frame-position (selected-frame) 70 70)

; Sets initial window size to 85 columns and 47 rows
(set-frame-size (selected-frame) 85 47)

; Makes last line ends in carriage return
(setq requre-final-newline t)

; Sets Ctrl-x / key combination for easy commenting
; out of selected lines.
(global-set-key "\C-x/" 'comment-or-uncomment-region)

; Allow resizing of the mini-buffer when necessary
(setq resize-minibuffer-mode t)

; I know lot of you will criticize me for this but all this
; years spent on Windows platform made me to do this:(
; Turns Windows-like selection mode.
(pc-selection-mode)

; Auto magically read compressed files
(auto-compression-mode 1)

; Set standard indent to 2 rather then 4
(setq standard-indent 2)

; This tells Emacs to create backup files.
(setq make-backup-files t)

; And this will enable versioning with default values.
(setq version-control t)

; Remove annoying message about deleting excess backup of .recentf
; which is list of recent files used and will be explained in next
; post.
(setq delete-old-versions t)

; Finally do not spread backups all over the disk.
; Just save all backup files in this directory.
(setq backup-directory-alist (quote ((".*" . "~/.emacs_backups/"))))

That’s it for now. This will make your Emacs a little bit better with nicer font and with backup files grouped in one directory.

In the next post we’ll add some useful extensions.

Blog at WordPress.com.