A Serverless Wiki on Windows

I began looking into serverless wikis when my server-based wiki at work crashed. I figured a server-less wiki would be a more robust, controllable way to store my information. Thus I discovered TiddlyWiki, a serverless JavaScript wiki implementation. TiddlyWiki is amazing because it's a single HTML file. Stick it into Visual Web Developer (or Visual Interdev - the keyboard shortcuts are the same), hit Cntrl-Alt-T and navigate the Document Outline. Here's a quick run-down of the TiddlyWiki source code. The head of the document contains a hashtable of title and version data, followed by copyright and content-type META tags. Next comes a CSS STYLE tag. The second last SCRIPT tag defines the config hash-of-hashes detailing configuration information for the wiki page.

Code Snippets Manager

Visual Studio Code Snippet files (.snippet) are managed by the Code Snippets Manager available in MSVC 2005, 2008 and Express editions). The keyboard shortcuts you need are:

Cntrl-K Cntrl-B To enter the CSM
Cntrl-K Cntrl-X To insert a code snippet in your code

Code snippets look like this:
<?xml version="1.0" encoding="utf-8"?>
<code xmlns="">
...some metadata and code...
</code snippet>
</code snippets>


Architecture Research at Microsoft

Jack Greenfield's article on "The Case for Software Factories" and his book"Software Factories" on amazon. This is excellent reading for anyone interested in RAD-related thinking. In Greenfields paper, he quotes research from the Standish Group, a group specializing in IT value research.

Also drop by MSDN's Architecture Center, currently involved in multi-core and cloud-related ideas. An interesting presentation on cloud can be found here.

Windows Vista Snipping Tool

A great improvement over XP's Alt-PrintScreen, this superb tool is a great boon to all screen-capture enthusiasts out there. To run it just type "snip" in the search box on the Start menu.

http://windowshelp.microsoft.com/Windows/en-US/Help/d87ecb6c-c1a8-487b-a381-dab33b3d102a1033.mspx#EGAAC

Excel Survival Guide

If you see dashed lines in your spreadsheets....
Tools->Options->View and uncheck Page Breaks (Alt-K)

If you want colored borders: Cntrl-1 around the cells to border up and select the color.

Alt-TI -> make sure "Analysis ToolPak" and "Analysis Toolpak" - VBA are added. This will bring in some useful worksheet functions. Examples are:

• EDATE(start_date, months). Note on dates: MS Excel stores dates as sequential serial numbers, so they can be used in calculations. By default, Jan 1 1900 is serial number 1, and Jan 1 2008 is serial number 39448. MS Excel for Mac uses 1904 as start date.
  
• Alt-TDD (Tools, Data Analysis). Select analysis methodology e.g ANOVA single factor.
  

Excel 2007 is Weird

http://www.theregister.co.uk/2007/09/26/excel_2007_bug/

Multithreaded Server Development in C#

ThreadPools - typically have a maximum number of threads. If all threads are busy, additional tasks are placed in a queue till one thread becomes available. If no threads are available, create a new set of threads to fill the thread pool. Some classes to consider:

ThreadPool - contains a HostProtectionAttribute (introduced in .NET 2.0). Here is some problem code using ThreadPool.

public static void Main() {
ThreadPool.QueueUserWorkItem(new WaitCallback(ThreadProc)); // Queue task
// Thread.Sleep(1000);
Console.WriteLine("Main thread exits."); }

The code above is bad as the program exits before ThreadProc runs (race condition).

WaitCallback - a callback method (delegate) to be called by a thread pool thread
AutoResetEvent - notifies one waiting thread an event has occurred
ManualResetEvent - notifies one or more waiting threads that an event has occurred.

If there are parts of the server you want to code in C++ you can create a C++ DLL using the Win32 Application Wizard. Here are the options you need to set:

• Select Application Type->DLL and Additional Options->Export Symbols
• If calling from non-Unicode code, do Project->Properties (ALT-F7), Configuration Properties->General->Character set (set to use multi-byte character set)
• ALT-f7 -> Configuration Properties -> C/C++ -> Code Generation -> Runtime Libray: “Multi-threaded Debug (/MTd)”. DLL will statically link to the C runtime library. It will increase the size of the DLL but DLL will not be dependent on the correct version of the C runtime library DLL. This is the default anyway.

CRT Library Blues

CRT libraries (Cruntime) are available in 3 distinct flavas in Win32 SDK.

One single-threaded and two multi-threaded variants.

• libc.lib, statically linked simplicity for single-threaded programs
• libcmt.lib, multithreaded cousin
• crtdll.lib, belle of the ball, an import library for CRTDLL.DLL (part of NT) that supports multi-threaded programs.

CRT redist package can be downloaded here, which in addition to CRT and Std C++ runtime components, also includes stuff for OpenMP (shared-memory parallel programming)and MSDIA.

Great Book on Black Box Testing

Written by Dr Boris Beizer.
http://www.soft.com/QualWeek/QW2001/Board/beizer.html

StreamReaders, StringWriters, MemoryStreams, FileStreams, Serialization in .NET

StreamReader - read files as lines of strings
FileStream - read files as arrays of bytes (need an Encoding to tranlsate back to strings)
MemoryStream - reads data from memory instead of from file

System.Runtime.Serialization
System.Xml.Serialization

Example of serialization of List:

TextWriter writer = new StreamWriter(PATH);
XmlSerializer serializer = new XmlSerializer(typeof(List<long>)) -- typeof returns an object of type Type
serializer.Serialize(writer, mylist);
writer.Close();

StringWriter - stream that can morph into a string
Path.GetTempFileName - create temp file and return full path

Over-Aggressive IntelliSense

IntelliSense can be a productivity boon - provided you don't need to undo rubbish autocompletes. Sadly bad autocompletes are very common. To minimise the frequency of false completions do the following:

Go to: Tools->Options [ Text Editor->C#->IntelliSense]
Go to: Committed by typing the following characters: [everything except letters and numbers!]

Remove . ) = ; } from the list.

Joys of TreeView

The first experience building a TreeView will take ages, the next not so long, and the next after that will be quick. Once you know the a) objects, b) various constructors, c) properties and d) events its very easy.

Creating hierarchy of nodes: TreeNode main = new TreeNode(); main.Name = "main"; main.Text = "main"; main.Nodes.Add(someNode);
Constructors from MSDN:
The TreeNode constructor has six different overloads.
http://msdn.microsoft.com/en-us/library/system.windows.forms.treenode.treenode.aspx
Notice how everything on MSDN is stored on ASPX (xhtml) pages.

Favorite Overloads TreeNode("text"); TreeNode("text", childNodeArray)
To get selected data: main.SelectedNode.Text
How do you reverse an array of TreeNodes? Array.Reverse(treeNodeArray);
AfterSelect event handler: processes a node selection. To process nodes of only a certain depth do: if (e.Node.Level == x) { do_something(); } where e is a TreeViewEventArgs object.

Remember how to program this using JTrees in Java?

Java is IRON, C# is STEEL.

Debugging 401 Errors in Internet Information Services

HTTP, an application level protocol for distributed hypermedia systems, came into existence in 1990 with HTTP/0.9, a simple protocol for raw data transfer over the Internet.

With HTTP 1.x came a built-in mechanism for authentication, fully documented in RFC 2617. HTTP 1.1 supports two modes of bulit-in authentication "Basic" and "Digest". Basic authentication transmits the username/password pair in unencrypted form from browser to server. It should only be used over an encrypted medium e.g. SSL (Secure Sockets Layer recently renamed by the IETF to TLS, or Transport Layer Security). Digest authentication sends a one-way hash of the username/password pair, calculated with a time-sensitive salt value from the server. The purpose of the salt value is that the username-password hash is always unique, and protected from replay attacks. Note though that only the authentication details are protected by the hash and any data sent following successful authentication will be visible to any party with access to the network traffic. To overcome this, an encrypted stream is needed.

Attempting to retrieve a page using HTTP authentication can result in an HTTP 401 (Unauthorized) error (some other interesting error codes are listed here, including the currently unused 402 error - Payment Required). The response from the server will include a "WWW-authenticate" header of the form (WWW-Authenticate: AUTH_SCHEME realm="REALM_NAME"). A valid value for AUTH_SCHEME would be Basic, to denote basic authentication. The "realm" is the section of the website protected by the authentication scheme. Since HTTP is stateless, servers don't remember who is logged in and clients must send correct headers for each protected page they wish to access.

IIS provides substatus codes to analyse the underlying causes of 401 errors. The codes are: 1) invalid credentials, 2) wrong authentication method, 3) ACL issue, 4) ISAPI (Internet Server API)/CGI issue, and, 5) access denied by URL authorization policy on server.

How to Write a Good Old Windows Batch File (GOTO considered useful)

Introduction

This post is about effective batch file writing. No Windows programmer is complete without a working knowledge of batch files. Here are some reasons you would use good old-fashioned batch files as a preferred technology over newer, shinier alternatives:

Advantages of Batch Files

1. Scripting languages are not installed on client PCs, doing so would be a hassle
2. Need to run some commands on a network drive which may have partial trust issues if implemented in .NET

Tips and Tricks

1. For comments, use :: instead of REM. The reasons will become apparent. : is a notation for labels in Windows batch files.
2. When setting variables do: set VAR="". Every time you use VAR you need to say %VAR%. That's pretty obvious when you're echoing stuff to the screen, but once you start to get into conditional statements, it can be easy to forget the necessary %% delimiters e.g. you need them when doing: mkdir %VAR%, or IF EXIST %VAR% GOTO VAREXISTS:, or COPY %SOURCE%\*.* %TARGET%\. See how ubiquitous percentage signs are in real-life batch file writing!
3. To access command-line arguments use %1, %2, .. , %n. Again the percentage sign - this time operating in a unary capacity!

@echo off (TRACE OFF)

Worth adding a word on the @echo off statement oft-seen atop many a batch file. echo off prevents commands echoing themselves irritatingly to the console; the @ prefix is a self-reference trick which applies the command to itself.

Tricky Syntax

Batch files have some syntactical tricks which seem a little awkward. Here is a summary:

:~n (colon-tilde) -> extract the first (or last if n is negative) n characters from a variable e.g. %PATH:~-10% will extract the last 10 characters from the path.

:~m,n (colon-tilde-comma) -> a small variation of colon-tilde. m characters offset, then next n characters. m=0 means extract all the characters. This is a substring function.

%~dp0 (percent-tilde-drive-path) -> %0 determines where the batch file is running from (a la %1, %2 et al) and the dp produces the drive letter and path - so this tells us which dir/path we are running from. %~dp1 will do the same for variable 1 (seems a bit weird to do that).

%~sp0 (percent-tilde-short-path) -> same as dp0 only returns "short path" (i.e. drive letter is missing so C:\WINNT becomes \WINNT)

Google groups has a post identifying even more % codes; take a gander here.

Bits and Bobs

To compose powerful batch scripts you need a good working knowledge of DOS commands that are legal under Windows. The site here gives a good Vista-focused introduction.

DirectX - The Next Episode

In August 2008, MS released their new DirectX SDK (compatible with XP as well as Vista) which includes Direct3D 10. In it there is a new "Games for Windows Branding Tool" to help test games versus "Games for Windows" test requirements.

Here is the home page for DirectX.
http://msdn.microsoft.com/en-us/directx/default.aspx

You can't call yourself knowledgeable about Windows programming if you don't know DirectX. The tutorials requires a basic familiarity Win32 programming such as the Windows message loop architecture (translate/dispatch).

DXUT is layer built on top of Direct3D to make tools easier to build. It simplifies the process for creating a "device" and processing Windows messages. Other tutorials focus e.g on creating a wave effect using a vertex shader.

Finally, if you think DirectX is just for games programmers think again. WPF, Microsoft's new GUI technology, renders graphics in DirectX via milcore.dll.

Joys of DGV -> DataGrid "Grande Vitesse"

There are multifarious ways to populate a DGV. One way is to create a BindingSource and then set that to be the datasource of your DGV. DGV uses what has been appelle'd the "standard WinForms data binding model".

If you have been using the old DataGrid of yore, you need to familiarise yourself with the diffs between DataGrid and DataGridView. DGV adds features missing from the mere Datagrid, understand bound and unbound data.

What MSDN says about "wor friend" DGV est (SCooBee DOO, DOO = dgv):

DGV is Truly Scalable!!! We can go from SMALL, read-only data views to LARGE EDITABLE views of data.
DGV is Truly Customisable!!! You can programmatically create your own CUSTOM SORTS and create your own types of CELLS. Yahoo! Awesomable customisable grid here we come!
DGV is Truly BINDABLE! (more on this amazing BINDABILITY property later!)

It will bind like a MAGENTRON to objects implementating:

1. Immortal IList interface of System.Collections fame (wa-hoo, that includes one-dimensional arrays! I am so pleased!)
2. AWESOME IListSOURCE interface, of System.ComponentModel fame (for "bindable" lists), implementes IList incidentally, the key question being does it offer any additional beneficiences? no major differences, except naming convention... just seems like this is the wrapper version for WinForm components...DataTable and DataSet implement the IListSource interface.

There are also to other classes capable of binding nicely to the DGV, they both have BINDING in their worth-mentioning names:

1. IBindingList interface.
2. IBindingListView interface.

Key Question - what are the actual physical mechanics of data binding to the DGV?

Most of time, you will create a BindingSource component containing most of the details of connecting to the data source. The BS component can represent any WinForms data source (WA-HOO)!

Make sure binding source for DGV is set programmatically and not via the IDE (you will have problems).

Style-wise, dgv.AutoResizeColumns should be used with caution, especially if you have set "preferred" widths for your columns.

Good Post on Enums as Ints

Highlights differences between enums in C++ and C#. Essential Reading.
http://blogs.msdn.com/peterhal/archive/2005/08/01/446357.aspx

Here's also a useful snippet showing how to iterate through enums in C#, useful if you are populating a ComboBox for example:

foreach (string s in Enum.getValuesOf(enum)) {}

Incremental Linking

Many a linker and build problem can be fixed by simply doing a build clean and repeating the compile-link-manifest cycle in MSVC.

But I recently found a problem in managed DLL compilation where object files were getting rebuilt but not correctly even after clean builds.

I traced the problem to the linker not refreshing some new code. Turning off incremental linking fixed this right away. I knew MSVC was using this heavily since my faulty builds were very quick and slowed down (but were 100% correct) when built without incremental linking.

Visual Studio Shortcuts

For those who use MSVC every day, shortcuts are "bread-and-butter".

Cntrl-MO Collapse ALL
Cntrl-ML Expand ALL

Shift-ALT-Enter In/Out of full screen mode

Shift-F5 Exit debugging

F12 - go to definition of identifier under cursor

Principles of C# Composition

#region William Strunk Jr, Principles of Composition (applies to programming as to prose)
"Make the paragraph the unit of composition: one paragraph to each topic".
#endregion

#region Windows Joe, Principles of C# Composition

1. When storing dates in data structures use DateTimes rather than one of multiple string formats. There are special situations when it is preferable to use strings e.g. when all the strings in the application are guaranteed to be in the same format.
2. Don't make assumptions about objects being reference or value types. Apart from user-defined classes and obvious things like ints, many objects in the .NET framework are inconsistently implemented - they may be values or references. ADO.NET in .Net 1.1 is an example. Use properties to get and set these variables explicitly rather than rely on reference-value semantics.
3. Use object adapters. Avoid class adapters.

#endregion

Gregorian, Julian or Korean?

To be an AWESOME Windows programmer you need to be able to manipulate calendars effortlessly, whether they be Gregorian, Julian or Korean. You should not need to rush to the nearest Internet access point to know the differences between Gregorian and Julian calendars.

Unfortunately the Calendar math of .NET is not so user-friendly especially when compared to recent additions to Python, for example.

Here is a small program that uses the Calendar class of Python 2.5 to generate dates for all weekends between 2000 and 2020. As you can see, the Calendar class is highly user-friendly (with its use of iterators) and also very ISO-friendly.

import sys
import datetime
from calendar import Calendar

def getdays(year):
c = Calendar(0) # creates a calendar with firstweek=0 (0 is Monday)
for j in range(1, 13):
for dt in c.itermonthdates(year, j):
isoweekday = dt.isoweekday()
if isoweekday == 6 or isoweekday == 7:
print dt.isoformat()

def main(args):
for year in range(2000, 2020):
days = getDays(year)

Beat that, .NET!

.NET possesess an eponymous Calendar class in its System.Globalization namespace (perhaps more appropriately named System.Localization, in the humble view of Windows Joe). In fact, .NET is so obsessed with Localization e.g. of date-time formats, that they completely ignore the fact that international ISO standards exist.

So the challenge - how do we program the same thing in C#? (Answer in the Comments section).

LogDisplayName: A Case Study in Windows in Object-Level Security

Surely awesome Windows progammers don't need event logs (a feature of NT since 1993, only relatively recently been seen in retail Windows) since they can program their own application logging without much fuss. Still, knowledge of the Event Logging API (exposed in .NET's System.Diagnostics namespace) is a prerequisite before once can be deemed a Windows master programmer. Hacking event logs also illustrates interesting points regarding "secure methods" in Windows.

Let's see what event logs look like in XP and Vista.

EventLog objects have "secure" methods (LogDisplayName)

Write a small program using System.Diagnostics.EventLog.GetEventLogs() method and logobject.LogDisplayName property to find and display event logs to the user. You will be flooded with SecurityExceptions! Here are the reasons!

1. Running code from an untrusted location (e.g. untrusted UNC path) - this results in the following: System.Security.SecurityException: Request for the permission of type 'System.Diagnostics.EventLogPermission' (insert culture and public key token info here) failed. Copying the assembly locally worked, but there were still exceptions.

2. Protected Registry Access - my locally-running program worked fine for Internet Explorer, Hardware Events and some others but not for My Computer. The error was: System.Security.SecurityException: Requested registry access is not allowed. The answer lay in the registry itself. Right clicking on "Eventlog" in the registry (below) revealed all.

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Eventlog

This idea of "secure methods" in an API resemble the "object model guard" on the Outlook API. This prevents client programs e.g. automatically sending email on behalf of the user, without explicit user consent. A buzzword for this concept of "secure methods" is object-level security. A rather old but useful post on SecurityException debugging .NET can be found here, on the .NET security blog. "Secure methods" are also configurable in Java via the SecurityManager class.

How to Hunt for Easter Eggs in Python Open Source Code

first steps with -m pdb

Conceive a simple code block with your favourite open source library then run:

python -m pdb simple.py

Type s and hit return repeatedly to step through the code. This gets boring after 10 seconds and you'll want to do more stuff. Type help and you'll see a slew of commands.

• j [linenumber] e.g. j 10 -> you need to be a good guesser here, you'll have a problem if you jump before or after the current code block
• commands - if your program is crashing, go into pdb, type commands and hit return and you will get a magic stack trace
• cont - restart the debugger if you're at the end of traipsing through your program

quit will exit the debugger. Now explore the other commands and make a list of your top 5 favourite debugger commands.

bugs as easter eggs

pdb is a great (though basic) for bug finding. Many people find bug-finding frustrating and gripe when difficulties come their way.

Think of bug finding as looking for Easter eggs. You understand a system well when you know all the Easter eggs hidden within it.

When faced with a new bug - celebrate! Difficulties will teach you much more than random success and "first-time lucky" code - you won't learn much from those kinds of experiences. Debug patiently, study the bugs, understand fully why they happen.

real-life easter egg examples

Here's a real-life Easter egg example I found in Python's BeautifulSoup (quick introduction - Python's BeautifulSoup (needs Py2.2 upwards) has two parts: BeautifulSoup for html, BeautifulStoneSoup for xml/sgml (built on sgmllib); it has a simple object model 1) PageElement - base class for Tag and NavigableString, and 2) NavigableString - base class for CData, PI, Comment, Declaration).

Grep for "enterprise" and you will see the Easter egg!

Here's another example where the above techniques were used to find a bug in a test program f0r BeautifulSoup, where the error message alone was insufficient to find the problem. I've copied the rogue code here for your perusal. Can you find the error using pdb? (you will need to install BeautifulSoup first obviously!). Here's the code:

import sys from BeautifulSoup import BeautifulSoup
def main(args): try: html = open("file", "r").readlines()
soup = BeautifulSoup(html)
soup.prettify()
print soup
except Exception, e: print e
main(sys.argv)

Look at the code above. What is it trying to do? What precise data transformations are taking place?

This is a good example of the caution needed when plugging the output of one API as input to another API. You need a clear understanding of the data types you are dealing with, and sometimes the internal represenation of those types (e.g. concept of "blittability" in .NET). It also shows that in dynamically typed languages you sometimes need to think even more carefully about data types.

A related post on UbuWorld entitled "Dynamic Typing Doesn't Mean You Don't Have to Think About Types" explores this idea with some more examples when "type-consciousness" is vital to debugging dynamically-typed programs.

We struggle with bugs when we forget programming is not magic but a very precise art form that is heavily dependent on exact data representation for things to work together correctly. Abstraction and dynamic typing (Scylla and Charybdis or programming languages) hides complexity but abstractions break when they are assembled in unexpected ways e.g. across API boundaries as illustrated above. Cost of such simplifications is that they make debugging more difficult, by giving the illusion of simplicity. Under the covers, very precise data transformations and memory allocations and deallocations are taking place. Don't forget this. Programming is a very precise art form.

Hint: the above example requires a one-line change to get it to work.

Happy Hunting.

Save as type: Web archive, singe file (*.mht)

You can save web pages as mht files in Internet explorer. What are the key benefits?

"Manifestation" of Managed DLLs

Managed DLLs are "born" with embedded XML manifests. Learn everything immediately on the Manifest Generation page or risk mediocrity in your Windows skills and knowledge.

Managed DLLs are sometimes referred to in Visual Studio circles as "metadata files".

Flying without Intellisense

As seasoned Windows hackers are well aware, opening an MSVC project on multiple machines LOCKS IntelliSense. Conceptually it's like flying without an HUD just like Luke Skywalker in the original version of Star Wars!! This gave me a great training idea...practising coding without IntelliSense. You need to know your APIs well, let me tell you! It's like chi sao blindfold! There are many benefits I can see with this training method:

1. creates an empathy for the programmers of yore who had no auto-completion
2. forces you to learn api method names accurately ....avoid conundrums like is it length or size...hmmm....I need to troll through all the auto-complete suggestions to find out which. thus it is a time-saver in the long-term.
3. forces you to design api names that are easily memorable and fit for purpose. You will also be tempted (probably) to shorten complex template names with typedefs, something you probably do already, but without IntelliSense you will be forced to embrace these disciplined habits!
4. In certain mixed-mode programming tasks, IntelliSense may need to be disabled for performance reasons
5. sometimes the .pcb file is locked and you can't use it anyway

If you want to be an AWESOME visual studio programmer (and who doesn't) try flying without IntelliSense from time to time. It's also more physically demanding to type all those method names (think of IntelliSense as the "power steering" of Visual Studio) but it's worth it.

Ghost on Windows

Download and Setup

Download a Windows build of GPL Ghostscript 8.63. It's a 10meg download (most of the memory being taken up by gsdll32.dll which is about 8.6MB). It is written by PDL vendor Artifex software. (PDL = Page Description Language). Also useful to grab is the graphical interface GSView. This install into C:\~PF\Ghostgum. You will need to add the following to your path:

1. C:\~PF\gs\gs8.63\bin
2. C:\~PF\Ghostgum\gsview

Error/Info Messages when Opening PS files

1. Displaying non DSC file - DSC=Document Structuring Conventions, set of instructions Adobe wrote to maintain consistent PostScript language between drivers and platforms.

Learn Postscript

Find out more about PostScript the product and PostScript the language here.

ActiveX Killbits in Vista

A new security patch from MS relates to killbits.

Programming C++/CLI (VS2005/2008)

Lack of Automation, Plethora of Design Choices

C++CLI (formerly but sometimes still referred to as "Managed C++") is a potent technology for making C++ code intelligible from C# client programs. Fortunately or unfortunately, .NET does not have any javah style tools that can automatically generate managed C++ headers for a given c# source file. S0 for now, you have to write your headers manually. But Managed C++ also allows for much greater flexibility than Java in writing managed/unmanaged interfaces and thus more variety of designs are possible.

The main keywords in C++/CLI are as follows.

1. ref -- as in "public ref class" declares a managed class or struct.
2. hat operator -- e.g. in return types such as System::String^, denotes a handle to an object on the managed heap (this is a separate heap maintained by the CLR and implements asynchronous gc).
3. percent operator -- this denotes a tracking reference and can apply to ValueTypes (e.g. int%) or reference types (e.g. String^%) to denote a modifiable parameter passed by reference.
4. gcnew - when returning managed objects e.g. return gcnew String("kill germs"); or, if converting from an STL string, you might do something like this: return gcnew String( stlString.c_str() ).

To compile a C++ program as managed C++ you need to use the /clr compile flag. (You can compile flags in Project Properties -> Config -> C++ -> Command Line).
Arguments, Return Types and the Blittability of Data Types: Strings and Arrays are non-blittable!
To make sense to C#, all return types must be CLR types and denoted as being on the managed heap using the hat operator. Inputs, though, can be C data types e.g. pointers, which are legal in C# provided pointers to stack allocated objects are used in an unsafe context, and pointers to heap-allocated (and hence managed) objects are pinned (i.e. not shifted around by the garbage collector). It is a design decision whether to allow unmanaged data as inputs into your managed C++ interface.
Certain data types are blittable across language boundaries, the standards ones for C++/C# are enumerated in this article on blittable and non-blittable types. These include System.Single, System.Double, System.Byte, System.IntPtr but NOT System.String, System.Array or System.Char. STRINGS AND ARRAYS REQUIRE SPECIAL CODE WHEN MOVING BETWEEN C# AND C++!!!
Core C# Keywords for interacting with C++/CLI are as follows:

1. fixed - used to create pinning pointers. multiple pointers of the same type can be "fixed" in the same clause e.g fixed (byte* src=array1, dest=array2) {...}. Otherwise use nested fix statements. "fixed" is just syntactic sugar for GCHandle.Alloc(pointer_into_some_array, GCHandleType.Pinned).
2. GCHandle - see fixed, relates to handling a managed type from unmanaged memory. Four kinds of GCHandle can be created, a pinned handle (commonly used), a normal handle, Weak handle and WeakTrackResurrection. GCHandles are allowed to be value types, the reason for this is that the lifetime of a GCHandle is controlled by Alloc and Free and not the garbage collector. Don't get too excited though - ARRAYS of GCHandles still need to be created on the managed heap!

Examples
Also, take a look at this msdn example. Two other fundamental concepts in Managed C++ are pinned pointers and interior pointers. Interior pointers are pointers into the CLI heap which point to managed objects (or members thereof) and thus have a magical "dynamic" property which normal pointers do not have (and do not need), namely tracking objects as they move through the heap. As we will discuss later, care must be taken when converting between interior pointers and pinned pointers!
System.Array class
It is worth knowing this class well, since it is the base class for all arrays in the CLR! Avoid provoking errors like "Add is not a method of System::Array". Also, as we know, arrays do not transmit seamlessly between C# and C++. Let's get down to business. Array is a "ref" class and an "abstract" class - only instantiable in its derived form. Despite this you can't derive from Array directly. If you try you get the error "cannot derive from special class 'System.Array'". Useful methods and properties:

1. Length (property). 32 bit integer representing number of elements. For a multidimensional array, call it d, to find the number of elements allocated for the first vector, we need d[0].Length. For 64 bit we need to use LongLength.

Multidimensional Arrays, Two Phrases
In C# the basic syntax for defining a (rectangular) multidimensional array is as follows:
int [,] myArray = new int[2,2]; myArray[0,0] =1; myArray[0,1]=2;
But this is not the only syntax. There is also a syntax for jagged, or non-rectangular, arrays. It looks like C++ (although you can't define arrays in C++ as below, you need subscripts, and brackets following the variable name rather than preceding it, but that's by the by):
double[][] locationCodes = new double[2][]; locationCodes[0] = new double[3];
These jagged arrays can be represented in Managed C++ using the templated form of the Array class: array<array<double>^>^ locationCodes = gcnew array<array<double>^>(num_arrays);
Can I pin a multidimensional array?
The short answer is no. A multidimensional array contains System.Arrays, which are non-blittable types and .NET DOES NOT ALLOW PINNING OF ARRAYS OF NON-BLITTABLE TYPES (GCHandle will generate a run-time exception). Pinning an array of primitive types is not a problem.
If you pin an array of arrays by a) pinning the subarrays b) pinning the main array (using a pinning pointer, as GCHandle won't allow pinning of arrays of non-blittable types) your program will compile. However, passing in your MD array from C# to Managed C++ then forwarding to a native C++ function (which copies the arrays) may result in a System.AccessViolationException: Attempted to read or write protected memory.
Let's work on the idea of pinning subarrays for a moment. How do we store the pinned pointers? In an array perhaps? An STL vector of pinned pointers will create pointers to pinned pointers, this is illegal - "pointer to pinned pointer disallowed by CLR". A native array defined using [] is illegal too - since "a native array cannot contain a managed type". A managed array (array<pin_ptr<double>>^ is illegal - "a managed array cannot have this type". Neither can you have an array of references to pin_ptrs because both pointers and references to pinned pointers are illegal. The bottom line is you can't store pinned pointers in a collection. What you can store, though, are GCHandles. So, to create an array of pinned pointers, the trick is: gcnew an array of GCHandles, use Length to find the number of pointers in the array, loop from i=o to maxdimensions, then use GCHandle::Alloc to pin the vectors in each dimension. You will then need to use pin_ptr to pin the array of pointers (GCHandle won't work since it doesn't pin Arrays of non-blittables).
GCHandles and pin_ptr are synergistic. What can a GCHandle do that a pin_ptr cannot do? Answer: GCHandles can be stored in collections. What can a pin_ptr do that a GCHandle cannot do? Answer: a pin_ptr can point to an array of a non-blittable type.
Copying Arrays
Given the complexities of using pin_ptrs and GCHandles, an easier alternative to interfacing with C and C++ functions is to simply copy managed arrays into their unmanaged equivalents. Here Marshal.Copy proves useful. (Marshal is one of the core classes in InteropServices).
Marshal.Copy copies data from managed arrays to unmanaged pointers and vice versa. Only problem is - it's designed for one-dimensional arrays. To copy multidimensional arrays into unmanaged memory, you need to write your own custom marshaller. This is easy provided you know how to program in C (knowledge of simple facts like - malloc returns void*, I need to cast it to a pointer to my specific data type e.g. int* or double*). Knowledge of C# is not enough, you need to know BOTH C AND C++ to be an awesome interop specialist. There is no restriction on the use of malloc and free from within managed C++ classes - use them to your advantage.
C++CLI MindMatter
Herb Sutter's blog - http://blogs.gotdotnet.com/hsutter/. Herb Sutter is the cool dude who brought us GOTW and joined MSFT's Developer and Platform Evangelism Division in 2002 to create more awesome C++ experiences in Visual Studio. Another very good post on malloc, new and custom memory allocation (useful in MC++) can be found here. And another interesting blog post on design choices in interop here.
C++/CLI is also formalised in an ECMA standard, number 372.
C++/CLI Programming Style (typedefs and pointer conversions)
If you decide to use typedefs for managed types e.g. typedef pin_ptr <const int>s; pin_constint32; typedef pin_ptr<const long> pin_constlong; the types used for managed and unmanaged code must exactly match the function signatures. For example, a pinned pointer to an int is not equivalent to a pinned pointer to a long, which is not equivalent to a pinned pointer to a const long, which is not equivalent to const long long. For interop to work, matches must be exact...otherwise you will get a lot of errors converting between interior pointers to pinned pointers.
Differences between C++ and Managed C++ (C++/CLI)
Can't supply default arguments to member functions of managed type, C3222.
Care with the Delete Operator
Calling delete twice on the same pointer officially results in undefined behaviour. In MSVC it results in the message: "Debug Assertion Failed! Expression: _BLOCK_TYPE_IS_VALID(pHead->nBlockUse). Calling delete on null pointers is safe though so it is good to set pointers to NULL after delete.
A double delete may happen in a ref class if a destructor and finalizer both delete the same unmanaged pointer.
Deterministic and Nondetermistic Finalization
"ref" classes in CLI extend IDisposable (automatically) and call the (deterministic) destructor when obj.Dispose() is called.
Criticism of CLI
There are a number of objections to CLI.

.NET Data Binding Architecture

BindingSource
BindingNavigator

The Data Binding Idiom is a Good One.

But let's say I have a dozen controls all on one form, each with different binding requirements. How scalable is this metaphor? Do I create new BindingSources for each one?

Classic C# Programming Offences (includes Managed C++ Gotchas)

1. Not declaring your constructors as explictly public (unless of course you are programming a singleton) [minus 10 points]
2. Trying to resize a text box vertically, when the multi-line property is set to false. [minus 5 points]
3. Trying to click on the toolbox in Visual Studio when you're not in Design View. [minus 9 points]"
4. Looking up the "Editable" property in a control when the correct property is "Read-only" (old school!) [minus 3 points]
5. Using \n in a text box and expecting a newline (you need \r\n) [minus 5 points]
6. Trying to pin the memory of a stack-allocated object like so: double d=5; fixed (double* ptrd = &d); you will get the error "cannot use fixed to take address of an already fixed expression". [minus 10 points - shows a fundamental lack of understanding of .NET memory management]
7. Trying to create an array or vector of pinned pointers. [minus 5 points]
8. Using new instead of gcnew in a Managed C++ program [minus 1 point]
9. Returning null instead of NULL from Managed C++ (reversed for C#) [minus 5 points - you don't know MS C Compiler]
10. Using ref to pass a reference type rather than a value type [minus 20 points]
11. Trying to add a Scrollbar to a Panel (use AutoScroll property!) [minus 2 points]
12. Writing a callback function as private.

Exploring DLLs using PE Explorer (I recommend the Export Viewer!)

I've just downloaded a 30-day trial version of PE Explorer from HeavenTools to see how well it can disassemble DLLs. On startup, load your DLL then press Cntrl-M and Alt-S to start the disassembler.

Cntrl-F for "entry" which takes you to the DLL entrypoint which should look something like this:
push ebp
move ebp, esp
etc

EBP is the (32-bit) frame pointer or the base pointer, unlike the stack pointer ESP, the base pointer is only manipulated explicitly. ESP always points to the last element used on the stack (NOT the first free element).

Cntrl-E takes you to the Export Viewer.

Visual Studio Multi-Targeting Support

MSVC 2008 can target .NET framework 2.0 as well as 3.5, Woo hoo!!!
http://weblogs.asp.net/scottgu/archive/2007/06/20/vs-2008-multi-targeting-support.aspx

Getting your Head Around Layout Management in Visual C#

Layout management is tricky business in Visual C#. People have written Java-style layout managers to address this problem. Take a look at Matthew Johnson's article. According to MSDN, you can achieve great layouts using just Anchor and Dock properties (not totally true). Anchor locks your control in one region of the form, Dock is similar but allows for dynamic resizing of the control. So what can MJ do that Anchor and Dock cannot do? BorderLayout (NSWE, Center), GridLayout, FlowLayout. Not satisfied? Hungry for more on Layout Managers? Read another blog about Layout Managers this time focusing on WPF. Hey, he missed out VirtualizingStackPanel from System.Windows.Controls! (the "control classes" are at the "core of the user's experience with any application" (Quote: MSDN)).

How good is DevExpress for Visual Studio .NET?

Devexpress have a bunch of .NET components (you need to download their 175MB DXExperience8.2.4 exe file to try it out). If IIS is not installed, you'll see an error "cannot create virtual directories". Don't worry about that.

XtraCharts suite - Harvard Graphics style pie charts (2d and 3d), donut charts, Gantt charts
XtraPivotGrid - integrates with charts, nice!

Embrace the API: Programming Model Re-engineering (aka Aggressive API refactoring)

Forget Refactoring. Meet your new friend - Programming Model (aka API) Re-engineering - Microsoft-Style. Microsoft are veterans in re-inventing existing programming models. Here are some examples, and related jabberings:

Flex versus WPF

Despite the pain of adapting to new programming models, libraries need to adapt to change, which means PEOPLE (consumers) also need to change. This brings me on to my next point concerning PEOPLE'S ATTITUDES to programming model re-engineering.

"The difference between greatness and mediocrity in Windows programming can be perceived in many ways. One metric is the speed by which the Windows Programmer can quickly adapt to radically different programming models. The Great One will be able to assimilate new APIs massively quickly and start programming in them straight away, no matter how intractable and problematic the API may initially seem. On sensing a new API approaching, the senses of the Windows programming are immediately sharpened, instinct kicks in and a process of lightning-fast power-learning is initiated. The millions of programs the Great One has written are instantly referenced and put into the context of the new API function calls. Automatic memorisation commences. Like an avid art collector, the SuperPower Windows Programmer desires to acquire API mastery swiftly and add to their ever-expanding skills collection. The SuperPower Windows Programmer may even be willing to embrace functionally-inferior but more modern APIs . Contrast this with the mediocre programmer, their emotional response is a negative one, a slow process of gradual learning and conscious and pained memorisation commences. Thus SuperPower Windows Programmers are quantum leaps ahead of their sub-par rivals. They are like eagles soaring above, surveying everything, their rivals are like meerkats down below, constantly digging up the dust to hide from predators".

Mantra-Based Programming and the "Programming Athlete"

Here's a great article on athletic performance which I think is very relevant to programmers. Read the article in reverse. The section on negative emotions affecting the athlete due to "concentration loss" is a typical one, particularly towards the end of a long coding session when tiredness has already set in.

If you feel your coding session is going haywire take the following actions. Consciously control the flow of your thoughts, don't act/code anything without feeling you are in a state of calmness and complete control over your coding. Stop time in your own mind. Stop being a whirlwind, breathe and see the bigger picture.

Use mantras to refocus your mind.

Then go back and finish your program.

Prototyping Numerical Algorithms in Windows - Excel as a Viable Alternative to Matlab

Excel spreadsheets and VBA are good tools for prototyping numerical algorithms. There's just a few things you need to know to get the most out of the experience, a few tricks and a few traps. Once you have the prototype, you can implement the results in a more industrial-strength framework (I like C++/C#).

Writing a Custom Worksheet Function in MS Excel (Alt-F11, Alt-Q)

Excel is a remarkable product that can be customised. Starting with the basics, everyone should know how to write a custom worksheet function in Excel. Example: Write a worksheet function that returns 2*PI.

Alt-F11 (make sure the cell is not in edit mode)
Function TwoPi() As Double
TwoPi = 2 * Excel.WorksheetFunction.Pi
End Function
Alt-Q

Excel worksheet functions only recalculate when their inputs change. If you change the definition of the function, and the arguments remain the same, the function will not recalculate with shift-F9. I discovered this bug when I tried to change a function definition dynamically at run-time, using a set of cells to specify coefficients. Changed the def, pressed shift-F9 and ... nothing happened.

Macro Security (Digital Signed Macros)

If your macro security level is set to high you will only be able to run signed macros from trusted sources. Alt-TMS will allow you to change your settings. Alt-TD in the VBA editor will help you set digital signatures for your macros.

Option Explicit

When developing algos in VB Option Explicit is your friend. Avoid creating variable names implicitly, which can lead to difficulty debugging your algorithm.

Function Overloading in Excel VBA Macros

This is not allowed. Any attempt to do so will result in "Ambiguous Name detected". I think this is ok for the simplicity of the language. You can get round this by putting version numbers on your overloaded functions.

Name Lookup Bugs in Excel 2002

Create custom worksheet function. Call it F. Rename it to F1. Rename it to F. #NAME error. Rename it back to F1. It works. May need to do a few iterations before you see the bug.
There are other name lookup errors that can occur which are not bugs. For example, editing a function in a module in worksheet A and testing it out in worksheet B will not work. Each worksheet is effectively a separate namespace.

Min/Max in Excel VBA

Amazingly VBA doesn't have a min/max function!!! So use Excel's instead (example: Application.Max(5,2)).

Random Number Generation

In Matlab you can use the function rand(10) to choose a number between 1 and 10. To do this in Excel you can use the function RANDBETWEEN(1, 10) but this relies on Analysis ToolPak being a valid add-in in your Excel. If you don't have access to Analysis ToolPak you can write the following worksheet function:

Function Random() as Double
Randomize: Random = Excel.WorksheetFunction.Ceiling( Rnd * 10, 1)
End Function

Custom Worksheet Functions in C#

Msdn Blog Post on Excel C#.

Programming Addicts

England's Guardian runs an awesome story on exercise addiction, called "Kicking the Habit". Could the same psychological affliction affect modern-day computer programmers?

Furthermore, are some programmers addicted to both programming AND exercise? What drives people to such addiction?

"Exercise addiction" is a term coined by Dr William Glasser , the "Warren Buffet of psychiatry", in a 1976 study of long-distance runners. He identifies a negative addiction when a SINGLE DAY away from the gym or pool causes distress. For some programmers, perhaps a day away from programming causes distress, hence we see the parallel's with Glasser's study.

But Why? Is it the endorphins of exercise, the joy of achieving something or just the impulse of crazy competition taking control of our psyche?

Originally a chemical engineer, Dr Glasser went into psychiatry when he realised that was his real interest in life. Who can blame him? Psychiatry is an important and fascinating discipline! As Donald Trump famously states: "Don't Waste Your Life on Work You Don't Love; Passion Will Help You Do Better". Read the blog of The Donald here.

In the 5th Century BC, psychotic traits were considered supernatural in origin (see wikipedia).

The Relevance of Charles Petzold's Programming Windows in a Modern .NET World

Charles Petzold has released a new book on 3D Programming using WPF. He is best known for his classic tome Programming Windows (using C and the Win32 API). But is this "Windows programmers bible" still relevant in today's world of .NET? Windows Joe takes a look....

Some might say that the Mighty Programming Windows using C and the Windows API is a topic only of historical relevance. Certainly the modern .NET programmer has no business using the Windows API in day-to-day programming ...in theory. What happens when an API function isn't exposed in the .NET framework. Well, we could just expose it manually in .NET and be done with it ....do we really need to know what's going on inside, or read a thousand plus pages to work out Windows' inner workings?

It all boils down to one thing. Do you WANT to be the best Windows programmer in the world or don't you? If the answer is an unwavering, unhesitating YES then you NEED, I repeat, NEED, to read Charles' book on Programming Windows quick-time. Don't delay.

Charles talks about the history of Windows and facts every aficionado of the operating system should know of the top of their hat - starting from Windows 1.0 "tiled windows" release in November 1985, the year "Iron Mike"/"The Baddest Man on the Planet" made his professional debut in Albany, NY, moving on to Windows 2.0 in 1987 (overlapping windows), the year the NY Giants defeated the Denver Broncos in the SuperBowl, then Windows 3.0 in 1990 (support for 16-bit protected mode in 286 and upwards), the year 300,000 people in Lithuania demonstrated for independence.

In April 1992, the legendary Windows 3.1 was released (a few months later in September, Black Wednesday took place, knocking the pound and lira out of the ERM). Hello TrueType fonts, OLE (remember OCX controls?) and standardized dialog boxes. Then Win95 in 1995 and Win98 in 1998. Charles summarises that the "Windows API has remained quite consistent since Windows 1.0" except in the area of function calls: "Windows 1.0 supported fewer than 450 function calls; today there are thousands". He also underlines the differences between Win16 and Win32. If you don't know about this, read the book.

The first program, which starts with #include windows.h (a header that includes a bunch of other headers like winbase.h for kernel functions) is the canonical start file for a Win32 program. Ah, the joys of WinMain!

Good Programming Blogs (not just OK ones)

Check out Eric Lippert's Fabulous Adventures in Coding Eric is on the C# 4.0 Design Team and has worked on script engines previously.

Also check out this list of the top CLR bloggers.

Why Aaron Removes Admin Rights

Talk about kernel-mode rootkit phobia!

Aaron Margosis the "anti-admin" of Microsoft wants to take away the freedom of operating Windows with full admin rights. The core of Aaron's expurgation is that though we may feel we use our machines responsibly, "things happen". Stay away from me, kernel-mode rootkits!

(Kernel-mode rootkits ARE the root of evil though. Imagine being able to access any address and execute any instruction. Yikes!)

Recently Microsoft's remove-power-from-user attitude has been giving me UAC grief in Vista. Exhibit A: A referral was returned from the server" and application fails to start! (VISTA) This indicates a UAC denial. What was I trying to do? Hack my own mailbox p'raps? Well, if you want a program to run with elevated privileges in Vista, it must be marked as such in the application's manifest file. This applies to managed .NET code.

Read up on UAC in Vista here.

Do Androids Dream of Dalvik VM?

I can tell you categorically that Androids DON'T dream about Dalvik VM.

Android apps, however, are built to run on a Dalvik VM (named after a fishing village in Iceland) which implements most of core Java and runs on Linux kernel version 2.6 (currently the latest version). If you are still running 2.4 you need to upgrade ASAP to 2.6 (which includes NUMA enhancements for multiprocessor, hyperthreading support for intel P4 amongst other things) to run Dalvik VM. Don't delay.

For app developers use Eclipse or IDE of choice (e.g. NetBeans or IntelliJ) on Windows as your Java/Android devstudio.

A common question - what are the differences between Dalvik and "traditional" JVMs. For one thing, java.awt with its great AlphaComposite class is notably absent, instead there is some import android.graphics.*; business going on. java.applet is also absent. Have a look at this example of android-style Java.

Java for Androids.What's next?

What's next is Scandaldroid - in other words - Scala on Android. Scala, the functional, object-oriented hybrid can use all the libraries of Java and C#.http://groups.google.com/group/jvm-languages/browse_thread/thread/828d946b01a4e3f2

scalac android.scala, folks.

Also worth checking out is the Google Android blog and a WordPress blog on Android.

What is Android?

We all know Google and mobile are synonymous with Android, but what IS it? Find out here.

.NET Framework 3.5 SP1 is Released

SP1 makes it easier to build REST clients and services.
A .NET Service Pack upgrade a day helps you work, REST and play.

Fancy Alt-tab

Windows Vista comes with a new alternative to Alt-tab --- the Windows-key tab -- a more graphical context switching tool that gives Vista a Linuxy L&F.

Wardriving

Read this wiki article for an introduction to wardriving.

Wireless Channels

I've found the need to switch wireless channels on my WiFi network regularly to maintain QoS. Fortunately, in Europe we have as many as 13 channels to choose from, whereas the US has only 11. The Intel Centrino chipset is also built using FCC-approved wireless which means only 11 channels.

This limitation also means that the number of channels available for public hotspots in Europe is effectively 11, to cope with the greatest possible number of users (e.g. American tourists won't be able to use channel 13).

Installing new objects in your home may increase multipath effects (basically radio waves bouncing on and off objects enhancing or diminishing the received signal) and may create the need to switch channels. Also new wireless networks in your area may require you to switch channels.

DEC Alpha Emulation for Windows

A Dutch company has developed PersonalAlpha, a DEC alpha emulator for 32-bit Windows (XP etc). Just as Linux is basically UNIX for PCs, PersonalAlpha is DEC alpha for the PC. 

Don't know what is DEC alpha or why PersonalAlpha is so cool? 

DEC alpha's run the OpenVMS OS (VMS=virtual memory system) developed by DEC (which was purchased by Compaq before being merged into HPQ) which has a reputation for solidity. 

Armies and stock exchanges use VMS technology (just check out this article here). Need a quick intro to programming concepts for VMS. I have just the ticket right here on the HP website.

Python Eggs

The Philosophy of Eggs

An increasing number of Python programs are being distributed as "runtime eggs". Eggs are simply a way of distributing Python programs similar to RPM. They are installed using easy_install.

An example of a package using eggs is Pyglet, a toolkit for games and visual effects (BSD-license).

A great deal of eggs can be found on the Python package index. They are installed in one of two ways:

easy_install package.egg
easy_install http://www..../package.egg

easy_install is based on the distutils package. Basic concepts of distutils can be found here.

The "Old Way"

Most packages currently come in tarballs that you unzip and then run:

python setup.py install

This will build and install everything in one step. If the build process is heavy you may want to split this up into two steps:

python setup.py build
python setup.py install

To install modules locally (in your own personal "stash") you need to do:

python setup.py install --home=<home>

Administrator Command Prompt in Vista

In Vista you will not be able to copy files into C:\Program Files....unless you run Administrator Command Prompt!!!!  Right click on cmd.exe and select Run As Administrator.

Add/Remove Programs in Vista

This feature has been renamed "Programs and Features".

Windows Side-by-Side Assemblies

What are WinSxS assemblies? You will find them in the C:\Windows\winsxs folder in Vista/XP. This folder is also known as the "native assembly cache".

My Vista install has over 6000 SxS assemblies (ranging from WinHTTP control to GDI+). According to MSDN, these assemblies are typically single DLLs described by a manifest (although an assemly can also bunch together DLLs, Windows classes, COM servers etc). The manifest describes the assembly AND its dependencies. I also have an XP PC which has only 1400 SxS assemblies.

From XP onwards, applications can use multiple versions of a single SxS assembly.

But what is the point of an SxS assembly?

An SxS assembly is effectively a DLL - a DLL with metadata described by a manifest - that is used by the Operating System as (to quote MSDN) "a fundamental unit of naming, binding, versioning, deployment, and configuration".

This MSDN page provides a flavour of the kinds of side-by-side assemblies that are out there.

I would define an SxS assembly as "an operating system DLL with dynamic version control". So at run-time a running program can use multiple versions of the same DLL simultaneously.

Paraphrasing, SxS assembly is a way of versioning DLLs, hosting multiple versions on the same system i.e. same machine and OS, and with versioning of course comes deployment, how do we deploy different versions of DLL on the same system, and how does our software know which version to use? It's a core part of the Windows platform.

Embedded Scripting in Windows Applications

There's a whole host of embedded scripting technologies in commercial Windows products.
It will be interesting to see how they evolve, especially in the move from VBA to VB.NET.

MULTIAPPLICATION PROPRIETARY SCRIPTING
VB.NET
Visual Studio

VBA (of course, you can also use COM to control these applications via VB.NET)
MS Office

SINGLE APPLICATION PROPRIETARY SCRIPTING
VisualLisp
Used to automate AutoCAD

PerfectScript
WordPerfect (also supports VBA)
Here are some examples of PerfectScript macros (you will see they are a lot like batch files).

OPEN SCRIPTING
Lua - Promixis (based in Santa Barbara, CA) have a product called Girder (home automation software, control lighting, projectors etc) which uses embedded Lua scripting (smaller memory footprint than Python).
Python - Used in OpenOffice (although this is perhaps used more on Linux)

We'll keep tracking the evolution of Windows automation on this page.

Inside the MSI File Format

How do Microsoft Windows installer (.msi) files actually work?

msi files began life in 1995 as a Microsoft project codenamed "Darwin" (note: Darwin is also the name for the X-Box 360's motion controller developed by Motus Games - not related!!!). What advantages do .msi files have over simple xcopy?

The key idea behind msi/Darwin is application installation as a transaction.

.msi files are essentially structure storage files and are best analysed using orca.exe. This is part of WinInstaller SDK but some websites will let you grab the individual file.

What's on my C: drive in VISTA??!!

C:\>dir grep -v DIR

gives a listing of all the files (extensions: bat, log, sts, exe, iss (InstallShield Silent Response file), ini, txt).

The exe is junction.exe from sysinternals, a "Windows junction creator and reparse point viewer". A junction is a directory symlink in Windows, aka NTFS "junction". Reparse points are the means by which Windows symlinks are implemented.

If you do dir/a you will see all the JUNCTIONs as well as DIRs!! In my system, C:\Documents and Settings is a JUNCTION pointing to C:\Users.

DNS flaw discovered

Dan Kaminsky, a researcher with IOActive discovered a bug in DNS which allows hackers to conduct "DNS cache poisoning".

Read more on Dan's blog here.

Internet Time Synchronization in Vista

After Vista synchronised my time settings from time.windows.com (which points to pool.ntp.org) my time zone changed!!! Reset by right-clicking on the clock in the bottom right hand corner, and clicking "Adjust date time".

Purify/Quanity - The Rational Story - But How Rational is Rational?

Rational is a software division of IBM that makes the Purify and Quantify products. Their products are built around the concept of Rational Unified Process (TM). Rational was founded by Paul Levy and Mike Devlin in 1981 and sold to IBM in February 2003, for $2.1 billion. On announcement of the deal, IBM SVP Steve Mills remarked: "Rational is an important element of our e-business on-demand initiative...a perfect complement to our existing four brands - WebSphere, DB2, Lotus and Tivoli."

Products Rational is known for:

• Purify
• Quanitfy
• Rational Rose
• Pure coverage

Read more from their library below.

http://www.ibm.com/developerworks/rational/library/298.html

Implementation of WPF

WPF code is rendered via MIL (Media Integration Layer). Calls in the MIL can be seen using depends.exe (dependencywalker.com), on milcore.dll (which interfaces with DirectX, latest version DX10 comes with Vista).

XAML is a very good innovation from Microsoft. They previously started separating interface logic from presentation layer using partial classes but turning the presentation layer into XML is another step forward.

.NET 3.5 doesn't provide tools for automatic XAMLification of old-school C# code. It upgrades Windows Forms source files but sticks to the Windows Forms way of doing things.

PDC 2008 in October in Los Angeles

The Microsoft PDC was where Avalon (WPF) was first announced in 2003. This year it's taking place at the LA Convention Center and will cover multi-core programming on Windows, Silverlight (MS version of Flash) and Charles Petzold presenting WPF and XAML. They will also cover F# and MS Cloud platform.

Find out more about F# here:
http://research.microsoft.com/fsharp/fsharp.aspx

Recursive Directory Listing

If you look at Java packages, you have a lot of nested directory structures. You need to be able to see subdirectory content and structure quickly.

Windows: dir /S [dirname]
Unix: ls -R [dirname]

The concept of "recursive flags" doesn't map directly to Windows programs.

Eclipse Ganymede Released

The Eclipse Project marches on with releases named after the satellites of Jupiter (IO, Europa, Ganymede, Callisto). The latest release has 18 million lines of code. It runs pretty fast with 2MB RAM.

1) Keyboard Shortcuts
The tricky thing initially (if you are used to MSVC) is to get used to a new set of keyboard shortcuts. Here is a quick summary to get started fast:

Cntrl-H search the workspace
Cntrl-F11 run
Simple F11 debug

When you debug (F11) it opens debug perspective.

F5 Step into (five-in)
F6 Step over (six-over)

The debug window at the top of the screen shows you which thread you are running in and which java runtime you are using.

2) Customizing the Editor
Alt-WP gives you the Window Preferences from where you can set line numbers. (General->Editors->Text editors, Show line numbers). Line numbers can be quite ugly but useful when debugging.

Windows Media Ad-Popups

Problem: Windows Media pops up every so often to display ads :-(. Go to IE, hit Alt-TPP and set the filter level to High.

It does mean you will need to explicitly allow pop-ups from sites like blogger.com.

Prevent Truncation of Page Borders when Printing

Format->Borders and Shading->Page Border->Options. Change "Measure from edge of page" to "Measure from text".

Text Box versus Frames in Microsoft Word

Both text boxes and frames are flexible containers for text; flexible in the sense you can reposition them and resize them. Frames historically were used when you wanted to wrap text around a graphic. These days you will probably use text boxes more often than frames. Use frames in special cases if you need to insert comments, footnotes, endnotes or fields like AUTONUM.  Text boxes can be converted to frames by clicking Format->Text Box->Convert Text Box to Frame (Alt-OOF).

WZC Hijack in XP

XP comes with a feature called "Wireless Zero Configuration" which provides automatic configuration for wireless adapters. I recently installed a new broadband service which came with a management utility that hijacks the WZC facility, making it very difficult to detect and configure new wireless connections. Consequently when you travel and move to a wireless network using different WEP keys the new network is not detected. To fix this go to control panel -> network connections. Right click on "Wireless Network connection" to see Properties, then the Wireless Networks tab and activate the checkbox "Use Windows to configure my wireless network settings". This will allow you to view all wireless networks and set WEP keys.

Control Windows Services

Sometimes you install a new program and it adds new services to Windows on startup. msconfig is great for configuring these services, but it doesn't let you interact (stop/start) services in an obvious way. To do that you need Control Panel->Administrative Tools->Services.

To make this function easily accessible (annoyingly it is even possible to configure Windows to hide Administrative tools) we should make a batch file and stick in a directory on our path, such as c:\bin. Here is s1.bat:

start C:\WINDOWS\system32\services.msc

Now we can use the command s1 to access services and start and stop them as appropriate.

If you are at all curious on what the default services are, such as Background Intelligent Transfer Service and IPSEC service, you may want to take a gander at The Overclockers Club.

Classic Mode Performance Advantages for XP

Switching to Windows Classic Mode in WinXP results in less intense graphics than the default display mode. To get the full benefits you need to change settings in two places:

1. Control Panel -> Taskbar and StartMenu -> Classic Start Menu
2. Control Panel -> Display -> Themes -> Windows Classic

Windows the way it used to be. Faster.

Windows Virtual Memory Paging File

When your system gets low on virtual memory (for example when you start running antivirus software), Windows will automatically try to increase the size of your virtual memory paging file. The idea is to supplement the physical RAM available to the system with "virtual RAM". The size of this virtual RAM is set by Windows to be 1.5 times physical RAM. Accessing the paging file makes more memory available but slows the system down due to disk accesses. It used to be possible, and often necessary, to expand virtual memory using 3.5 inch disks on Windows. You can't really do this trick with CD-RW or DVD-RW, which have much shorter rewrite cycles than all previous storage media.

Recording Internet Audio

Just downloaded the open-source program Audacity 1.2.6. I'm trying to export some recorded audio a) in WAV format (which warns me my tracks will be mixed down to a single mono channel) and MP3 (which requires lame_enc.dll - LAME being an MP3 encoder licensed under LGPL, see lame.sourceforge.net for more details). I haven't looked at the source code but Audacity just seems to record the noise from your speakers. If you redirect the audio output Audacity gets confused. There might be a way to configure this properly but it's not obvious.

Free Antivirus

I've been using EZ Antivirus from CA for a while and my license expired. Result? -lots of annoying dialog boxes.

Tip 1

Rather than renew I switched to the more easy-to-use and free-for-non-commerical-use program AVG (free.grisoft.com). Unlike EZ, AVG does the noble thing and adds itself to Windows "Add/Remove Programs" so you can uninstall it easily later if you wish (EZ Antivirus requires you to run the program unvet32.exe but it does a decent job removing registry entries, services etc). Word of warning - AVG's exe is 47 MB and may take you a while to download.

Tip 2

I  had to disable the LinkScanner which is a cool feature but was slowing down my web browsing I also had to disable 404 redirection in the browser.

Tip 3

If you have a scheduled scan there it doesn't ask you e.g. if you want to scan now or delay or reschedule the scan by 1 hour. This is a pain since it can be hard to interrupt a scan in progress. To deal with this, ensure the scan is set to take place at a time which will cause minimum disruption to you e.g. 2am.

There is also Avast! now which gives away anti-rootkit (a paid option with AVG).

Crontab style Scheduling in Windows using Python

Here is one approach to crontab-style scheduling in Windows:

http://www.kalab.com/freeware/pycron/pycron.htm

Another would be to use UNIX emulation with cygwin. Hmm...I wonder how pycron works. What benefits does it offer over scheduled tasks in the Windows environment?

Querying the Task Manager Quickly

Doing Cntrl-Alt-Del and running Task Manager is a time-consuming process. Instead, the tasklist command can be used to get a list of running tasks.

For example, if you want to get a list of tasks taking up more than 50Megs in memory then use the "FI" (filter) parameter with tasklist as follows:

tasklist /FI "MEMUSAGE gt 50000"

An easy mistake to make is to type ">" rather than "gt". This will not work. It will give you an error: "ERROR: The search filter cannot be recognised".

On my system this returns just one process:

Image Name PID Session Name Session# Mem Usage
firefox.exe 3120 Console 0 70,116 K

There don't appear to be any built-in sorting functions based on memory or CPU usage, but this is easy to program by parsing the output of tasklist.exe in Python, Perl or some other scripting language. You will need to use tasklist /V (for verbose mode) to see CPU usage information.

Manage Shortcuts in Start Menu using cmd.exe

Shortcuts in Start Menu can be easily accessed via two locations in cmd.exe:

C:\Documents and Settings\All Users\Start Menu
C:\Documents and Settings\Your User\Start Menu

This is useful e.g. if you want to quickly delete a shortcut, or duplicate shortcuts:

del "Windows Media Player (2).lnk"

Doing the same operation by right clicking and pressing delete can take up to 5s compared with less than 1s using the command-line.

Windows Shortcuts Demystified

http://www.i2s-lab.com/Papers/The_Windows_Shortcut_File_Format.pdf

What's interesting here is the hidden file attribute bit.

Quick Cut and Paste in a Command Window

You need to copy a region of the command window but you can't select what you want. You just need to use the right mouse button to cut and paste.

1. Right click -> Select Mark
2. Right click to copy
3. Right click to paste

Easy.

Improve CPU Performance using msconfig

Start->Run->msconfig This nifty utility allows you to disable programs and services that run on Windows startup. One example is realsched.exe, a RealPlayer update-checker that runs out of C:\Program Files\Common Files\Real\Update_OB directory.

Control the Windows Control Panel

In order to edit your environment variables in Windows really fast, you need control over the applets in Control Panel.

Copy the following code and stick in sys.bat

start C:\WINDOWS\system32\sysdm.cpl

Save sys.bat to your bin directory, e.g. C:\bin\sys.bat. Now you can access System Properties quickly.

Howdy

Howdy, Welcome to the blog of Windows Joe - your dedicated guide to Windows PC maintenance!