Chris Cant's developer blog

Using PHDCC CodeModule with javascript

2011-08-04T09:22:00.007+01:00

phdcc Director John Cant writes:

My use of DNN has become -

[a] Maintain any HTML in a text file / copy and paste it into the basic editor / convert to raw / save. (Online editors are too hard to use and keep slim)

[b] Put any functionality into a phdcc.CodeModule

Much valuable functionality - jquery-ui / google maps / facebook / janrain / plupload / galleria - is available as javascript libraries. To get them onto a DNN Page, I have created a template CodeModule as follows.

ASP.NET gives you limited control over where in the page javascript can be inserted - RegisterClientScriptBlock inserts somewhere near the top; RegisterStartupScript somewhere near the bottom. All I can say is - the template works on all browsers tested.

In this example, I load the jquery-ui library at the top, and daisy chain the DNN onload handler at the bottom. The javascript could of course be located in a separate file.

In the onload handler, I create an invisible empty jquery-ui dialog and initialise a jquery-ui accordion. I bind the submit button of a PHDCC form to a handler and examine one of the form fields to give feedback via the dialog.

I add the CodeModule somewhere out of the way on the page - the bottom pane is a good place. Once there, accordions and dialogs can be used anywhere on the page.

If you need any help with this or any of the suggested libraries, please get back to us.


<%@ Control Language="C#" ClassName="JCformWidgets" %>

<script runat="server">

//--------------------------------------
private void loadJS()
{
 string scriptKey = "JCjsIncludes:" + this.UniqueID;
 if( Page.IsStartupScriptRegistered( scriptKey) || Page.IsPostBack) return;

 ClientScriptManager cs = Page.ClientScript;
 Type cstype = this.GetType();

 // ----------------------------
 // load JS files at the top of the page
 // ----------------------------
 string scriptBlock = "\n";

 scriptBlock += "<link type='text/css' href='/jqui/jquery-ui-1.8.14.custom.css' rel='Stylesheet'>\n";
 scriptBlock += "<script src='/jqui/jquery-ui-1.8.14.custom.min.js' type='text/javascript'>";
 scriptBlock += "</";
 scriptBlock += "script>\n";  // avoid microsoft parsing bug

 cs.RegisterClientScriptBlock( cstype, scriptKey, scriptBlock);

 // ----------------------------
 // load onload handler etc. at the bottom of the page
 // ----------------------------
 string startupScript = 
  @"<script language='JavaScript'>
<!--

function JConload(){

 var oldOnLoad = window.onload;

 var myDialog = $('<div></div>').dialog( {autoOpen: false} );
   
 this.newOnload = function(){
  
  // call existing handler
  if(typeof oldOnLoad === 'function'){ 
   oldOnLoad();
  }

  // -------------------------------------

  $(function() {
   $( '#accordion' ).accordion();
  });
  
  // -------------------------------------

  $('#' + 'dnn_ctr422_ViewForm_btnSubmit').bind('click', function() {

   var theseSkills = $( '#' + 'dnn_ctr422_ViewForm_Group_157_400_Question_157_400_1563_form_157_text_1563').val();
   var thisDialogText = '';
 
   if( theseSkills.length <= 0) {
    thisDialogText = 'No skills given<br>Please try again.';

   myDialog.html( thisDialogText);
    myDialog.dialog( 'option', 'title', 'Things you forgot ...' );
  
    myDialog.dialog( 'open');
    return false;
   }
  });

  // -------------------------------------

 };

}

(function(){
 var JCOL = new JConload();
 window.onload = JCOL.newOnload;
}());

            // -->";

 startupScript += "</sc";
 startupScript += "ript>"; // avoid microsoft parsing bug

 cs.RegisterStartupScript( cstype, scriptKey, startupScript);
}

//--------------------------------------
protected void Page_Load(object sender, EventArgs e)
{
 loadJS();
}

</script>

Scheduled task ping for plesk in Windows

2011-05-12T11:17:00.000+01:00

I've been moving several domains to a Windows cloud hosting package at http://www.webhosting.uk.com/. I'm currently using Cloud Beginner hosting that gives 10 domains and a dedicated IP address for £12/month. This package lets us run DotNetNuke (DNN) sites - and fast too as there is not much contention on the included SQL Server databases. The cloud is UK based so the ping time is less from the UK and nearby.

The account is run using Windows plesk which lets you control most things. However intervention is often needed from the webhosting.uk.com staff. They are usually very responsive and competent on their Live Chat service. Typical requests include setting ASP.NET modify permissions for the Network Service user and setting Application Starting Points. You need to insist that you get the dedicated IP address if you've asked for it. Email is included using the Horde web client, which isn't great; perhaps using Google Apps for email might be the best solution. No web stats package is included. Nor are file or database backups.

For one domain, one task that I needed to do was to set up scheduled tasks to ping various URLs. While this could be done externally, I decided to use the Plesk scheduled task feature. When FTPing in, the main site is at /httpdocs/. The scheduled task VBS scripts should be placed in /cgi-bin/

Using a couple of resources on the Internet, I put together this script, called pingurls.vbs:

Call DoScheduledTask()

Sub DoScheduledTask()
 On Error Resume Next

 Const ForReading = 1

 Set objFSO = CreateObject("Scripting.FileSystemObject")
 Set objFile = objFSO.OpenTextFile("E:\inetpub\vhosts\mydomainname.com\cgi-bin\urls2ping.txt", ForReading)

 Dim URL
 Do Until objFile.AtEndOfStream
  URL = objFile.ReadLine
  if Len(URL)>0 then PingURL(URL)
 Loop
 objFile.Close

end sub

sub PingURL(URL)

 Dim RequestObj
 Set RequestObj = CreateObject("Microsoft.XMLHTTP")
 'Open request and pass the URL
 RequestObj.open "POST", URL , false
 'Send Request
 RequestObj.Send
 'cleanup
 Set RequestObj = Nothing

End Sub

This script reads a text file urls2ping.txt, also in the /cgi-bin/ directory. Each line contains a URL to be pinged.

How do you know what absolute path to use to get to the text file? This ASP.NET code, say in MyPath.aspx, will tell you the path it is running from:

<%@ Page Language="VB" %>
<script runat="server">
Sub Page_Load()
 lblPath.Text = Request.MapPath("~")
End Sub
</script>
<html>
<head>
 <title>Web app path using Request.MapPath</title>
</head>
<body>
 <asp:Label ID="lblPath" Runat="server" />
</body>
</html>

The final stage is to set up the scheduled task in plesk. This is straight forward using [Scheduled Tasks] then [Add New Task]. The Path to Executable file should be similar to that used above, ie E:\inetpub\vhosts\mydomainname.com\cgi-bin\pingurls.vbs. Set the times you want the task to run.

Installing Tomcat onto Ubuntu Apache2

2010-11-20T09:14:00.012Z

This post describes how to install Tomcat 6 on an Ubuntu 10.10 server and connect it up with the Apache2 server using the jk module. Although there are plenty of posts about this, none gave a full list of instructions; this is what I aim to provide - let me know if you have any corrections.

I assume that you have Apache2 installed.

You will need to do most of the following actions at a root shell prompt. Get this from your login using: sudu su - You will need to edit various text files. You can do this using vi at the shell prompt. However I usually do this on my local computer using my preferred editor, transferring files to and fro using FTP.

Installing Tomcat

This command gets Tomcat 6 - and Java if need be:
aptitude install tomcat6

I'm not sure if that gets all the Tomcat files, so enter this to get them all:
apt-get install tomcat6 tomcat6-admin tomcat6-common tomcat6-user tomcat6-docs tomcat6-examples

On my system, Java was installed to here: /usr/lib/jvm/default-java/

You now need to set up Java environment variables in text script file /etc/environment
Add this to the PATH variable: /usr/lib/jvm/default-java/bin after a colon separator. Add these lines:

JAVA_HOME=/usr/lib/jvm/default-java
JDK_HOME=/usr/lib/jvm/default-java

You will now need to reboot so these changes are used.

Tomcat should now be running. You can start, stop or restart it as follows:

/etc/init.d/tomcat6 stop
/etc/init.d/tomcat6 status
/etc/init.d/tomcat6 restart

or more easily like this:

service tomcat6 restart

Tomcat should now be running on port 8080 at your host, eg http://www.example.com:8080/ If you go there, you should see an It works page. To test it further, I put a JSP file in here: /var/lib/tomcat6/webapps/ROOT/ and saw that it ran OK. I then put a servlet WAR file in /var/lib/tomcat6/webapps/. This was soon expanded automatically by Tomcat and I was able to navigate to the servlet directory (still on the 8080 port).

You may find it useful to run the manager and host-manager Tomcat applications. To enable these you need to edit the Tomcat configuration files in /etc/tomcat6/. Edit tomcat-users.xml to include the following:

<role rolename="manager"/>
<role rolename="admin"/>
<user name="admin" password="secret_password" roles="manager,admin"/>

You should now be able to access these apps at /manager/html and /host-manager/html. You will need to enter your credentials; possibly a couple of times on first access.

The Tomcat logs are at /var/log/tomcat6/. By default Tomcat creates a different log file for each day. Keep an eye on these, and the disk space they consume.

Connecting Tomcat to Apache

Hopefully you already know these commands to test your Apache configuration and restart Apache. You'll need these later.

apache2ctl configtest
apache2ctl restart

First, tell Tomcat to listen out for connections from Apache. Edit /etc/tomcat6/server.xml to uncomment this line:

<Connector port="8009" protocol="AJP/1.3" redirectPort="8443" />

Restart Tomcat.

Install the jk module that acts as the connector between Apache and Tomcat:
apt-get install libapache2-mod-jk

You should see a file called jk.load in /etc/apache2/mods-available/. I edited this to contain:

LoadModule jk_module /usr/lib/apache2/modules/mod_jk.so
JkWorkersFile /etc/libapache2-mod-jk/workers.properties
JkLogFile /var/log/apache2/mod_jk.log
JkShmFile /var/log/apache2/jk-runtime-status
JkLogLevel    info

You can also add JkMount instructions here, but I found that I needed to put them in each VirtualHost file in /etc/apache2/sites-available/. Eg in my default VirtualHost file I added:

JkMount /*.jsp ajp13_worker
JkMount /manager ajp13_worker
JkMount /manager/* ajp13_worker
JkMount /host-manager ajp13_worker
JkMount /host-manager/* ajp13_worker

An alternative is to define JkMount commands in jk.load and add a JkMountCopy command to copy these settings to all virtual hosts.

You now need to edit the jk worker file /etc/libapache2-mod-jk/workers.properties. Make sure it has code as follows, ie to tell jk to use the Tomcat connection we defined earlier:

worker.list=ajp13_worker
worker.ajp13_worker.port=8009
worker.ajp13_worker.host=localhost
worker.ajp13_worker.type=ajp13

Restart apache.

I was now able to access the Tomcat manager here: http://www.example.com/manager/html.

Edit your other VirtualHost files to expose any other apps that you need.

You'll find the Apache and jk logs in here: /var/log/apache2/. As above, keep an eye on these files and the disk space they use.

If you see "missing uri map" in log file mod_jk.log then you need to define JkMount in your VirtualHost.

As a final task, if you wish, disable the Tomcat 8080 port by editing /etc/tomcat6/server.xml and then restart Tomcat.

Windows 7 system drive letters

2010-09-21T17:29:00.008+01:00

I've previously blogged about Wandering drive letters in Windows 7. I've now a little more hard-won knowledge of what's going on.

Be very careful if you resize a Windows 7 system partition when the partition letter is not C. In fact, don't do it!

I am revisiting this issue because I ran into a problem. Our family shared laptop came with Vista. However this started playing up showing a DOS box regularly and Windows Update wouldn't install a particular patch. So I decided to install Windows 7 on drive D. This worked fine with W7 running with its system drive as D:.

Recently, drive D: started getting full so I decided to resize the partitions with the great Partition Wizard. This seemed to go OK. However, Windows 7 started playing up very badly, showing DwmHintDxUpdate error messages and worse. I eventually tracked the problem down to the fact that Windows 7 had decided that drive D: would now be called drive C: which is a serious problem as Windows and other software will store many full file locations (in the registry and elsewhere). If W7 thinks a file should be on drive D when in fact it is on drive C, then there's going to be serious problems. It's amazing that it booted at all.

The post by MarcusOS7 here explained what was going on, ie that Windows 7 tries to rename its system partition to drive C if the drive is resized, ie when the drive identifier changes. This is a serious flaw in Windows 7 and must be fixed.

I had hoped that with this information, I would be able to mend my W7 system. Even though I could just run regedit, I did not know what magic to do to change the system drive letter.

Eventually I decided to reinstall Windows 7. I saved the data over to an external USB drive. For this installation I wanted to be sure that W7 would install itself on drive C. Previously I must have done the installation when running the Vista system which somehow prevented W7 from using the C drive. This time, I was careful to do the install after booting into the Windows DVD. This trick ensured that W7 marked its install partition as drive C (even though it was not on the first partition).

In fact: before doing this, I re-partitioned the system so I had 3 partitions in the laptop, one each for Vista and W7, and one as a data drive. This process also helped me clear the previous W7 installation which was quite stubborn to remove, permission-wise. I eventually had to clear the partition by reformatting the drive.

Back in the new W7 system, I was able to restore the desired user data from the backup. And then I had to set up W7 again from scratch for each user, and installing all my development tools. As usual one of my first tasks is to turn off various system sounds, particularly the annoying Start Navigation click...

Drupal staging site security

2010-08-25T22:21:00.002+01:00

Suppose you have a Drupal staging site where you are preparing to go live or testing new features. You could install this on a separate domain that you have. Here's how to redirect casual users to your live site, while giving those in the know easy entry.

The crucial trick is to use a Session variable to indicate an authorised user. All Drupal access is via the root index.php file (except use of static files). index.php is amended to redirect users who do not have the session variable set correctly. Another secret file eg password.php, is used to let you get into the site by setting the session variable.

In the following example, www.example.com is your live domain and www.example.info is the staging server. The following code is on the staging server.

In index.php add this code after the line that contains drupal_bootstrap...

if( $_SESSION['password']!='asecret')
{
  header('Location: http://www.example.com/');
  exit;
}

Create a secret file in the root directory eg password.php with content like this:

<?php

require_once './includes/bootstrap.inc';
drupal_bootstrap(DRUPAL_BOOTSTRAP_FULL);

if( $_SESSION['password']=='asecret')
{
  header('Location: http://www.example.info/');
  die();
}

$pwd = trim($_POST['pwd']);
if( get_magic_quotes_gpc())
{
 $pwd = stripslashes($pwd);
}
if( $pwd=='asecret')
{
  $_SESSION['password'] = $pwd;
  header('Location: http://www.example.info/');
  die();
}
?>

<html>
<body>

<form method="post">

Security:
<input type="text" name="pwd" />
<input type="submit" value="Go" />

</form>

</body>
</html>

Tool to find maps KML Lat/Lng

2010-04-23T13:59:00.002+01:00

Use this tool if you want to find a Google Maps KML GLatLng location, eg to use within your own code: http://www.phdcc.com/GoogleLatLng.htm Either drag the initial marker or enter a search term. The latitude and longitude values are listed below, to 6 decimal places of accuracy.

The map is centred on the UK, as this is where I will want to use it most. It uses the Google AJAX Search API to work out UK postcodes well. Look at the source to see how it works.

This is based on the code by cmarshall at http://www.webmasterworld.com/xml/3542700.htm

DNN5 User soft-delete issues

2010-04-19T12:45:00.005+01:00

In DotNetNuke DNN5 when you delete a user that login is no longer removed from the system. Instead they are soft-deleted, ie a new IsDeleted flag column is set in the UserPortals table. (Note that IsDeleted in the Users table is *not* set - is this ever set?)

In DNN5, code that calls DotNetNuke.Entities.Users.UserController.GetUser() etc will return a UserInfo object even if the user is deleted. Therefore you may have to check the UserInfo.IsDeleted property every time you get a user.

I would have not have implemented it this way. I'd keep the DNN4 functionality and have extra API calls to find deleted users. I wonder: does the DNN core code always check IsDeleted now?

Anyway, UserInfo.IsDeleted is not available in DNN4. As I do not want different versions of my code for DNN4 and DNN5, I have written this isUserDeleted() static method that uses reflection to detect if the IsDeleted property is available and if so calls it. It returns true if the UserInfo is null. After that, for DNN4 it always returns false.

public static bool isUserDeleted(UserInfo ui)
{
 if (ui == null) return true;
 try          {
  Type tUserInfo = ui.GetType();

  PropertyInfo piIsDeleted = tUserInfo.GetProperty("IsDeleted");
  if (piIsDeleted != null)
  {
   bool IsDeleted = (bool)piIsDeleted.GetValue(ui, null);
   return IsDeleted;
  }
 }
 catch (Exception) { }
 return false;
}

Freegle Find a Group mashup

2010-04-10T09:00:00.021+01:00

This blog describes how I've used one JavaScript script to power several different incarnations of a Google Maps tool to help people Find a Freegle Group. Usage instructions for the Find a Group tool are on the Freegle wiki. You can also see the Find a Group gadger lower down on this page on the right.

This is primarily a technical article - for usage instructions, see the above link.

Freegle is a network of local re-use groups in the UK. Group members give away and receive any unwanted items for free. This helps conserve the world's resources, clears clutter and helps others.

The Find a Group tool shows a UK map with pinpoint markers for each Freegle group. You can move around and zoom in as normal - clicking on a marker shows an information bubble - clicking on the link in the bubble takes you to the group web site.

Find a Group also has a search box. The tool uses Google to search for that location in the UK. If found, the tool zooms to that location and adds a marker - you are invited to look for the nearest Freegle group marker. The nearest groups are listed below

Source data

The green group markers come from this KML file - maintained by another Freegle member. This is the latest data so the tool should always be up to date.

Actually, that's not the whole truth. The tool needs to load the KML file into memory so it can search for groups. Loading the above Google Maps KML link doesn't work because of cross-site scripting security (as I understand it). Therefore the code loads a separate snapshot of the KML file from a URL that it can access.

There's a plan to resolve this issue by having the primary KML file on the iLoveFreegle.org servers. That way, my code would only need to reference one file. And the plan is to have this file generated directly from the main Freegle groups database, so it is always up to date. (Currently the Google Maps KML file has to be updated by hand.)

Mashup variants

(P) Full web page

The Find a Group tool occupies a full web page here: maps.ilovefreegle.org.

(M) Google Mapplet

The Find a Group tool appears in the Google mapplets directory here. When added to My Maps it shows filling the browser window.

(I) Iframe

The Iframe version of the Find a Group tool is designed to fit into a small window within another page. The usage instructions describe the various configuration parameters.

You can see the iframe in action at the Penrith and Eden District Freegle group and its associated helper web site.

(G) Google gadget

The Find a Group tool is packaged as a Google Gadget, defined here. You can therefore add it to iGoogle, add it to a blog, and get the code to put on a web site. The gadget has similar options to the iframe.

The gadget has a Content type of url, with the main gadget code here. The gadget options are passed as URL QueryString parameters.

(F) Facebook application

The Find a Group tool is being developed as a simple Facebook iframe application.

Common Page Layout

Each of the above tools has HTML which has the same page elements:

Map div called "map_canvas"
Noscript block
Div called "jsBlock" that wraps the search form
Input text box called "search"
Submit button that has an onclick handler "return doSearch();" that always returns false
Information div called "idInfo"
Debug div called "idDebug" that shows version info by default

Accessibility

The tool has a noscripts block that has a link to the main site groups list - this is shown if the browser has no JavaScript or it is turned off. However Chrome doesn't show this text if JavaScript is turned off - sounds like a bug to me.

The "jsBlock" search form div is not displayed by default; the JavaScript makes this block visible. This avoids confusion if JavaScript is not available.

The search text box has an accesskey of "4" defined. For the larger variants, a label is defined for this text box.

The tool HTML is hopefully valid XHTML and CSS.

API and api keys

The Google Maps/Mapplets API is used in the JavaScript. There are various differences depending on whether it is a mapplet or not. Therefore one of the JavaScript initialise() parameters is a mapplet boolean; this is stored in a global for later use throughout the code.

To use the Google Maps API, each site needs an API key. All these tools run on the maps.iLoveFreegle.org domain so the iLoveFreegle.org API key is used. The mapplet code does not need an API key.

JavaScript structure

The JavaScript is pretty straightforward procedural code, with some globals, an initialise() function and a function doSearch() that is called when a user does a search. The JavaScript code is here: http://maps.ilovefreegle.org/freegle_maps.js.

initialise() function

This takes parameter specifying whether big map controls are wanted, and the mapplet boolean.

If not a mapplet, check that the browser is Google Maps compatible
Get references to the "idInfo" and "idDebug" page elements - add the js version to idDebug
If the "jsBlock" page element is present, change its display style to block to make it show.
Get the map object and set the map height if it has been specified in a parameter.
Get the z SearchZoom integer parameter for later use

Set the map to be the approx centre of the UK at a suitable scale
Use GGeoXml to load the primary KML file and add it as an overlay - the code cannot access the contents of the overlay, so...
Initiate a separate load of the (secondary) KML for processing, either using _IG_FetchXmlContent() or GDownloadUrl()
Finally set the focus to the search box if required

processKML() function

This parses the received KML XML file and stores info about each Group placemark object in the Groups array.

Note that the KML file stores each coordinate the wrong way round, ie longitude, latitude then height. The GLatLng.fromUrlValue() function needs just the first two values but reversed.

Once the KML is loaded, it sees if a search has been requested and does it.

doSearch() function

The search function does a Google GClientGeocoder search and zooms in to show the result. The search also lists the nearest groups, calculated using GLatLng.distanceFrom().

doSearch() uses showAddressResult(), SearchGroups() and createMarker().

Helper functions

String.prototype.trim, PageQuery, queryString() and isInteger() were obtained from code on the Internet. I added queryString2() helper function.

I have updated PageQuery to cope with URL parameters with key=value pairs where "=value" is missing.

800700C1 KB977165 and verclsid

2010-03-19T19:41:00.007Z

Our Vista laptop has developed 3 problems recently, which are probably all related. Can anyone help?

Windows Update keeps on trying to install KB977165, sometimes successfully, but it keeps re-appearing.

Clicking on Check for Updates now fails with error 800700C1.

It also often shows an error with a DOS box title C:\Windows\system32\verclsid.exe and a message 16 bit MS-DOS Subsystem saying The NTVDM CPU has encountered an illegal instruction.

We did have some software for 2 HP printers installed. I've uninstalled one of them as this was suggested as being a problem.

Any other ideas pls?

Crystaltech to 3essentials DNN copy

2010-03-09T10:42:00.003Z

PHDCC Director John Cant describes the process of transferring a DNN site from one shared host to another, at this stage simply as a backup using a different/private domain.

The site, ourdnnsite.com is being copied from Newtek (Crystaltech) hosting to 3Essentials with domain ourdnnsite.info. Both support services are very helpful but it currently seems that 3essentials runs faster than Crystaltech, probably because of a faster database server.

The existing site runs only one portal but this contains many users and many files in /Portals/0/. Maintaining the UserIDs is crucial as much custom information in the database is keyed off this value.

Enable ASP.NET 3.5 at the 3essentials site.

First, place a holding page at ourdnnsite.info, eg index.html, that will be shown instead of Default.aspx for people accessing the root domain URL.

Make sure to truncate transaction log prior to the upgrade, clear the eventlog and maybe even sitelog and search tables, if those are exaggeratingly large.

The 3essentials site currently runs SQL Server 2005/2003. They need a .BAK backup file so that the database copy can be restored correctly.

Do a database backup on the Crystaltech site and ask for the entire site to be zipped, to get these files:

ourdnnsitedb_1002050754.bak
ourdnnsite.zip

Load these to the 3essentials site ourdnnsite.info /private/

Submit a 3essentials site support request, asking for:

the database to be restored from /private/ourdnnsitedb_1002050754.bak
the site files to be unzipped from /private/ourdnnsite.zip
the site file permissions to be set correctly:
"I have corrected this issue, this involves giving the httpdocs directory IWPD User - Modify permissions which you are not able to do via Plesk."

Check the Web.Config file at the 3essentials site:

set the new database connection string twice
check the ObjectQualifier
set CustomErrors to 'Off' to see the errors
ensure the machinekey values are the same as the original values from the Crystaltech site
set AutoUpgrade off

On the database:

Update the portal alias, eg:
UPDATE [DNN_PortalAlias] SET HTTPAlias='www.ourdnnsite.info' WHERE PortalAliasID=0

Access the site by visiting Default.aspx

If any errors are visible, keep deleting the sections of Web.Config that cause errors.

If you cannot log in, use Chris' technique, described here. To access the DNN database, log in using myLittleAdmin here: http://db1.3essentials.com/mla/

To prevent unauthorised access to the ourdnnsite.info site, add to the Default.aspx file a check that a cookie is set. Provide a means of setting the cookie using another private page.

There are a couple of other additions that we usually make:

As well as creating an app_offline.htm file, we use the Cookie technique in Default.aspx to ensure that the site is offline during maintenance, redirecting to an offline page.
In the skin, we warn people who are online that the site is about to go down; this also helps us see if anyone is online now.

More details on these Default.aspx techniques later if you wish.

.NET app using Jet on x64

2010-02-24T12:02:00.005Z

If you are trying to use an Access MDB file using the Microsoft Jet OleDb driver in .NET or ASP.NET, you must set your platform target to x86 (32 bit).

If you are running on a 64 bit system and your assembly target is Any CPU or x64 then you will see this exception:
The 'Microsoft.Jet.OLEDB.4.0' provider is not registered on the local machine.

The Microsoft ACE.OLEDB driver for MDB and ACCDB files works OK whichever platform is targeted. However this driver is not installed by default on user computers unlike the Jet driver.

This fix can be used to make existing code work with Jet/MDB, eg the Cassini server.

Useful blog links:

Force .NET application to run in 32bit process on 64bit OS by Gabriel Schenker

Back to Basics: 32-bit and 64-bit confusion around x86 and x64 and the .NET Framework and CLR by Scott Hanselman

System.String hidden UTF8 BOM

2010-02-22T14:04:00.006Z

In .NET, a string (System.String) can contain an initial UTF-8 Byte Order Mark (BOM) which might not be seen in ordinary processing but is present when converted to a character array or into an encoding byte array.

For example, a text file might be saved in UTF8 format with UTF-8 Byte Order Mark bytes at the start, ie 0xEF 0xBB 0xBF. You might receive this file in ASP.NET using a FileUpload control, or read it directly in a Forms .NET app in C#:

byte[] FileBytes = File.ReadAllBytes(path);
string content = Encoding.UTF8.GetString(FileBytes);

If the file contains these 7 bytes (in hex) EF BB BF 44 65 61 72 then content will superficially contain the single word "Dear", eg as seen in the debugger, and content.StartsWith("Dear") will return true.

However, content.Length is 5 and content.ToCharArray() will return an array with 5 elements, the first being set to 0xFEFF. Similarly, Encoding.UTF8.GetBytes(content) will return the same 7 bytes as was used in the first place.

(Note that that has nothing to do with the encoderShouldEmitUTF8Identifier optional parameter for the UTF8Encoding constructor.)

As this hidden extra character can be misleading, I have written the following snippet that detects the presence of the UTF8 Byte Order Mark preamble and ignores it if present:

byte[] FileBytes = File.ReadAllBytes(path);
int StartPoint = 0;
int Count = FileBytes.Length;
if ( Count>= 3 && FileBytes[0] == 0xEF && FileBytes[1] == 0xBB && FileBytes[2] == 0xBF)
{
 StartPoint += 3;
 Count -= 3;
}
content = Encoding.UTF8.GetString(FileBytes, StartPoint, Count);

PS The code could not doubt be improved using Encoding.GetPreamble()

Clearing nested CSS floats

2010-02-07T12:59:00.006Z

Here's a XHTML/CSS technique to ensure you only clear the desired floats and get the right background areas for your DIV.

The code examples are here: http://www.phdcc.com/CSS_ClearingFloats.htm - look at the source code for full details.

My starting point is to have a DIV set with floats left and right. Within the middle unfloated DIV, I have another DIV set with floats and right. I want to clear just the middle floats, not the outer ones.

The crucial trick is to add CSS style 'overflow:hidden' to create a Block Formatting Context. Any CSS style 'clear:both' then only applies to the 'nearest'/current Block Formatting Context.

An associated problem was the fact that the background of the middle unfloated DIV goes wide to the border of the current Block Formatting Context. Adding CSS style 'overflow:hidden' to create a new context constrains the background to the expected area.

Using 'overflow:hidden' feels slightly naff: shouldn't there be an explicit CSS style to define a Block Formatting Context, instead of using something that does this as a side effect? Also, you may be concerned about what overflow is being hidden - as long as you have the default width:auto and/or height:auto set, you will not be hiding anything.

Adding Access key support to DotNetNuke

2010-02-05T15:01:00.014Z

This page explains how to add "access key" support to a DNN5 web site.
Access keys provide keyboard short-cuts to improve web accessibility, as described here - which also lists the UK government recommendations:
http://en.wikipedia.org/wiki/Access_key

I have implemented access keys using the accesskey attribute rather than the
Role Access Model

How it works for a user

For standard users, each browser uses different modifier keys and acts minorly differently when pressed. Consider accesskey '1'. In Windows Internet Explorer, pressing Alt+1 selects the corresponding link - you must press Enter to go to that link. In Firefox, pressing Alt+Shift+1 goes to the associated link immediately.

These accesskey shortcuts take precedence over standard Window commands, so if you define an accesskey 'f' it will be acted on rather than show the File menu in Windows.

In Internet Explorer, the link is not selected if the link has a style of display:none.

Search accesskey

Most of the access keys can be defined in your DNN skin using extra HTML that is not normally seen. However, accesskey 4 takes you straight to the Search text box. To make this happen, I had to amend admin/Skins/search.ascx to add AccessKey="4" to the txtSearchNew asp:TextBox.

While there, I took the opportunity to add a label for this box, as recommended for all fields to improve accessibility. This label is present but not normally displayed:
<asp:Label AssociatedControlID="txtSearchNew" style="display:none;" runat="server">Search:</asp:Label>

Main accesskeys

The main site access keys can be defined using code like this at the top of your skin, which I adapted from another web site:

<ul id="skips">
<li><a href="#content" accesskey="s">Skip to page content, accesskey=s</a></li>
<li><asp:HyperLink ID="SkipToHome" NavigateUrl="~/" AccessKey="1" runat="server">Skip to Home page, accesskey=1</asp:HyperLink></li>
</ul>

CSS (below) can then be used to make sure that this information is not normally seen, but each link become visible when it has the focus or is active. The list-style is set to none to avoid bullet points etc. When not in focus, the links have width and height 0; when in focus these are set to auto. Tweak the other settings as you wish.

Finally, don't forget to provide an anchor within your skin for the skip to navigation link:
<a id="content" name="content"></a>

CSS code:

ul#skips { list-style: none; }
#skips li { list-style: none; display: inline; }
#skips a
{
 color: white; width: 0; height: 0; overflow: hidden; z-index: 1000;
 position: absolute; top: 15px; left: 100px;
}
#skips a:active, #skips a:focus
{
 color: red; border: 1px solid red;
 width: auto; height: auto;
 display: block; overflow: visible;
 padding: 3px;
}

DNN local install with SQL Server

2010-02-01T15:10:00.015Z

The post summarises the steps required to set up DotNetNuke 5 (DNN5) locally on Windows using the full Microsoft SQL Server using SQL Server authentication, ie not the Express version. Getting the Logins and Users right is the crucial trick that I want to remember.

Click on each image to see it larger.

In Microsoft SQL Server Management Studio, connect to your local server. Right-click on Databases on the left. Select "New database". In the following screen, enter a database name, eg "DNN522pen" and click OK without changing anything else.

Now, find the server Security and right-click on Logins and select "New Login...":

Enter a login name such as "dnn522pen", select "SQL Server Authentication" and enter the password. Untick "Enforce password policy" if you wish. Do not set the Default database. Click OK.

Open up your database tree on the left: open "dnn522pen" then Security then Users. Right-click and select "New user...":

In the new user dialog, enter a User name such as "dnn522pen" and the Login name you used before eg "dnn522pen". Tick "db_owner" twice below and click OK:

Next, create a new folder on disk somewhere, eg in directory "D:\dnn522pen\". Unzip the DotNetNuke community installation file in there DotNetNuke_Community_05.02.02_Install.zip.

Open up Internet Information Services (IIS 7) Manager. Expand the menu on the left to open Sites and Default Web Site. Click on "Add Application..." on the right.

Enter a suitable alias and physical path, eg "dnn522pen" and "D:\dnn522pen\":

In IIS Manager, select the new alias on the left, select Content View and click "Browse" to start the DNN installation:

After a delay you should see the first DNN install screen. Choose custom:

Test file permissions.

Now, set up your database connection:
- First, click on "SQL Server 2005/2008 Database" and wait for the screen to refresh
- Enter "(local)" for the Server
- Enter the database name, eg "dnn522pen"
- Uncheck "Integrated Security" and wait for the screen to refresh
- Enter the User ID and password
- Keep "Run as db Owner" checked
- If desired, enter a database table name qualifier, eg "DNN_"
- Click on "Test Database Connection"
- If OK, click on Next

Proceed with the rest of the DNN installation as normal.

Importing Identity column data into SQL Server

2010-01-15T15:25:00.007Z

Updated 23/1/10:
I am in the process of porting some (DNN module) tables from one Microsoft SQL Server database to another. For various reasons, I want the primary key identity column values to remain the same. This is on a shared SQL server, so we don't have admin access, eg to the command line and file system.

The SQL Server Import and Export wizard transfers the data nicely and will preserve the identity values if you do it correctly.

There are two crucial tricks:
- set up the destination table(s) with the correct primary keys etc before the copy
- in the wizard, click Edit Mapping and tick Enable Identity Insert.

----------------------
This is alternative technique which should no longer be needed:

I first export the original data to a temporary database. This creates the tables but but does not create primary keys, set default values etc. However, the copy process does preserve the original primary key identity column values.

By installing the DNN module on the destination system, the tables are created correctly there. However, a wizard import into these tables (even with "Enable identity insert" set) does not preserve the key values.

The trick I found is to upload the data into a new table eg "Form2", and then use the following code to copy the data into the correct "Form" table. The trick is to use the "set identity_insert" statement.

set identity_insert DNN_Form on;

insert into DNN_Form ([FormID],[FormName])
select * from DNN_Form2;

set identity_insert DNN_Form off;

You must list all the required columns in the insert statement.
Only one table can have identity_insert on at a time.

Finally, delete the "Form2" table.

You can check the current insert identity value before and after as follows:

DBCC CHECKIDENT(DNN_Form);

Wandering drive letters in Windows

2010-01-14T20:37:00.003Z

My main computer's motherboard died - long live the new computer! Thankfully, all the data on the two existing drives were OK. A hastily bought 3.5 inch USB drive caddy worked a treat, keeping me going on the laptop.

The new computer was pre-installed with Windows 7 Home Premium on drive C with a separate partition on drive D for data. I have an MSDN subscription and I wanted to set up various versions of Windows multi-booting on the same computer. I have two different systems that I want to run normally, one with Visual Studio 2008 and the other with various older bits of software. I also want to have some different versions of Windows for software testing, ideally Windows 7, Vista and XP, both in x86 and x64 versions.

OK - I know I can run a virtual PC, but I want to use two of these systems regularly at top speed. I haven't tried virtual PC yet, so I decided to stick with what I knew, which is the HyperOs multibooter, though I had to upgrade to the latest version.

The first task was to repartition the disk, to give more partitions, each with a Windows installation, as well as big photo/video partition. I found that there were already two extra partitions, a (Packard Bell?) Recovery Partition and a small System Reserved partition set up by Windows 7, ie 4 Primary partitions in total. The W7 Disk Management tool wanted to convert my disk to Dynamic Disks to give me more partitions, but these cannot be used for booting, so a swift exit was called for.

HyperOS mentioned the Acronis partitioner but this wasn't playing - not sure if it was Windows 7 or the size of the disk. I did have a copy of gparted on CD, but this hadn't worked for me before. Eventually I found Partition Wizard www.partitionwizard.com which has worked brilliantly. I got a free commercial licence for this.

Using Partition Wizard, I deleted (empty) drive D and made quite a few other Logical partitions. Partition Wizard is clever enough to know that it cannot do changes in some circumstances, and so does it on reboot. Partition Wizard can also resize partitions, moving data if need be. So far that's worked fine. (Check your computer power settings don't shut your disk down at an awkward moment.)

Having done that, I could now copy all my precious data onto the new hard disk from the drive caddy.

Anyway, I've now done various Windows installations, some by installing from DVD from Windows, but mostly by installing from DVD at reboot, ie choosing DVD at boot up from the BIOS boot menu - and choosing Custom Setup to choose the install partition. The problem that I have found is that the drive letters that Windows uses and sees seems to change a lot. The original W7 system is on drive C on the "first" partition. I have another W7 on the fifth partition which thinks of itself as being on drive M - fine. However, most other W7 and Vista installations think of themselves as being drive C, even though they are on partition 4, 7 or 8. When I have rebooted in one of these systems, the drive letters are assigned in a fairly random way. The DVD drive is usually drive E but not always.

In Windows Disk Management you change the drive letters - for some drives at least. But there's limited scope for what you can change to. Partition Wizard can do this, and is probably more successful. However there appears to be no way of persuading a Windows on partition 4 (that thinks it is at drive C) to think of itself as being drive L for example.

All these wandering drive letters might not be problem. However my software development stuff and business data has always been carefully set up (for various reasons) to be stored on both drives C and D. I don't tend to put data in "My Documents", "Documents", "Pictures", etc because these are (usually) stored in different locations for each version of Windows. Anyway, I have persuaded partition 2 to be drive D in all the installations so far. However it was a problem that partition 1 kept wandering all over the shop. My solution was to move all my crucial data from drive C onto a bigger drive D. Ok - fairly simple in itself, but I'm still having to work out what dependencies there are in all my scripts.

Another complication in this process was that Windows XP was dying during installation (with a BSOD). This turned out to be because the SATA drives were being accessed using AHCI. Changing the BIOS to use the SATA setting "Native IDE" got the XP installation to work. However I did not want to leave this setting as is, so I change it whenever I want to switch to XP. XP also doesn't recognise many of the motherboard peripherals, eg Ethernet, so the installation is not very useful. The option to press F6 during installation would let me install a suitable driver, but (a) I don't have the driver and (b) the system doesn't have a floppy; it does look as though new motherboard has a floppy interface, but there's no connector soldered in there!

I was also able to add an IDE/PATA cable and drive to the system to connect my old drives, but the installation still did not work if I was in AHCI SATA mode.

I've still many applications to configure and systems to set up, but I'm getting there.

Starting to program vtiger CRM

2009-09-25T11:49:00.017+01:00

This article applies to vtiger 5.1.0 - some of the advice/code may not be accurate for later versions.

We've been investigating the vtiger CRM as a component in an open source set of tools that a business or organisation might need. A crucial idea is to link various tools together, so I have been looking at how to get information into and out of vtiger.

As a start, we're looking at an Outlook add-in to add an email address/name to the vtiger Contacts list, and how to export to an accounts package. There is already an Outlook vtiger add-in, but it seems pretty flaky and only does synchronisation; we want it to check a new email for existing matches and only add it if need be.

About vtiger

vtiger is a Customer Relationship Manager (CRM). vtiger deals with Leads who might turn into Contacts, with a Contact associated with an Account. You can set up Products with stock levels (and Services as well) and then generate Quotes and Invoices which can be downloaded as PDFs or sent by email. All that sort of thing. It is important to know that vtiger is designed for use only by a business' staff - it is not intended to be used by customers, although vtiger can check incoming support emails.

vtiger is extensible and there is some basic documentation to get you started making your own modules. The instructions effectively describe how to create a new object type (eg a time sheet), represented as a table or two in the vtiger database. You can link your module and its new object type into vtiger, so that eg menu [Tools][Time sheet] shows a list of time sheets in the standard vtiger style, clicking on a record shows the detail, with an Edit option available.

For the rest of this piece, I'm looking at adding functionality to an existing module. This will hopefully help you even if you are starting a complete module. Be careful if you alter existing code - it may well be overwritten if you upgrade vtiger.

The vtiger database

You will almost certainly want to have access to the database that vtiger uses, partly so you can see what's in there, but also so you see any changes as they really are. For MySQL, using phpMyAdmin will probably be the available tool - make sure you can get in to your database.

If there aren't any already, make some test Contacts within vtiger. In phpMyAdmin have a look at these tables:

vtiger_contactdetails
vtiger_contactscf
vtiger_contactaddress
vtiger_contactsubdetails

Each of these tables has a column named 'contactid' or similar as a primary or foreign key. Note that the "contact_no" column is what is displayed is vtiger as the "Contact Id", eg 'CON1'.

The table vtiger_modentity_num is used when generating the next "contact_no" value, for the appropriate module, so 2 might be next free Contact number to make 'CON2'. Later, we'll use vtiger CRMEntity class setModuleSeqNumber() function to get this value.

The 'contactid' numbers identifying the contact don't start at 0 or 1. In fact, vtiger assigns a unique id number across all objects, so object 792 might be a Contact and 793 might be an Invoice. These details numbers are stored in the vtiger_crmentity table - a deleted flag in there is set when items are first deleted - so they can recovered from the recycle bin. Later, we'll use the insertIntoCrmEntity() function to get a new object id.

vtiger command structure

All vtiger accesses come through one root file index.php. The parameters determine which module operation is carried out. For example, consider this URL

/index.php?action=DetailView&module=Invoice&record=800&parenttab=Inventory

It runs the code DetailView.php in the directory /modules/Invoice/. The code that runs in there, finds the "record" value and shows the relevant invoice. So, you can add to the "Detail View" functionality by adding code to this file.

You can create your own actions by making a new file in the relevant directory. You then need to provide a means of invoking that action - more later.

Smarty templates

vtiger uses the Smarty template system to generate its output. By and large, the vtiger code doesn't generate any HTML - instead, it loads up a template from TPL files in the /Smarty/templates/ directory. vtiger PHP code sets various values that are combined into the template to form the final output. Smarty has an extensive control language available for use in a template, so it can cope with complicated structures, eg it can iterate through arrays to generate multiple table rows. One Smarty template can invoke another, so finding where something is generated can be tricky.

Adding a tool option to an Invoice

If you view an Invoice there are Tools listed on the right hand side: "Export to PDF" and "Send Email with PDF". To add another option here, you have to drill down through the tpl files: DetailView.php invokes template Inventory/InventoryDetailView.tpl which includes template Inventory/InventoryActions.tpl.

In InventoryActions.tpl, after this line:

<!-- To display the Export To PDF link for PO, SO, Quotes and Invoice - ends -->

add in this code:

{if $MODULE eq 'Invoice'}
{assign var=send_accounts_action value="SendAccounts"}
<tr><td align="left" style="padding-left:10px;">
<a href="index.php?module={$MODULE}&action={$send_accounts_action}&return_module={$MODULE}&return_action=DetailView&record={$ID}&return_id={$ID}" class="webMnu"><img src="{'actionGenerateInvoice.gif'@vtiger_imageurl:$THEME}" hspace="5" align="absmiddle" border="0"/></a>
<a href="index.php?module={$MODULE}&action={$send_accounts_action}&return_module={$MODULE}&return_action=DetailView2&record={$ID}&return_id={$ID}" class="webMnu">Send to Accounts</a>
</td>
</tr>
{/if}

Upload InventoryActions.tpl and view an Invoice - check that "Send to Accounts" is visible on the right.

Adding to accounts

Now, create a file SendAccounts.php - probably copying an existing file such as DetailView.php makes sense.

OK - this bit is skimpy for now, but it might get you started. Near the end, add in the following code to query the vtiger database and display a little info from each record found. To get the list of products ordered you will also have to query the vtiger_inventoryproductrel table.

$query="select * from vtiger_invoice where invoiceid=?";
$invoice_id = $focus->id;
$result = $adb->pquery($query, array($invoice_id));
$noofrows = $adb->num_rows($result);
$final_output = "";
while ($inv_info = $adb->fetch_array($result))
{
$invoice_no = $inv_info['invoice_no'];
$subject = $inv_info['subject'];
$invoicedate = $inv_info['invoicedate'];
$accountid = $inv_info['accountid'];
$total = $inv_info['total'];
$final_output .= "$invoice_no: $subject $invoicedate $accountid $total<br/>";
}
$smarty->assign("SEND_DETAILS", $final_output);
$smarty->display("Inventory/SendToAccounts.tpl");

At the end of our new code, the Smarty variable "SEND_DETAILS" is set to the string we want to display. Then the Smarty template SendToAccounts.tpl is run.

You had best create the template file SendToAccounts.tpl initially as a copy of an existing template, eg InventoryDetailView.tpl. You will need to pare it down - and eventually add in a line like the following to display your output:

<p><b>Storing data</b>: {$SEND_DETAILS}</p>

OK - all we've done so far is display the invoice details. Actually talking to an accounts system is beyond the scope of this article.

Adding a new contact

As I said earlier, I want to have an Outlook add-in that can be used to add a contact to the vtiger database, after checking to see whether the contact already exists. As the first step to getting this done, I set up a small web form that has First name, Last name and Email fields. The Go button is set up to go to this vtiger URL eg:

/index.php?action=AddOutlookContact&module=Contacts&FirstName=Chris&LastName=Cant&Email=sales@phdcc.com

As per usual, vtiger uses its standard rules so that this script is called:
/modules/Contacts/AddOutlookContact.php

I made AddOutlookContact.php in a similar way to that described above, using the title "Import from Outlook" for the main content tab. The code generates the output HTML in the PHP script - a neater final approach would put the raw HTML within the template. Anyway, the PHP code generates its HTML in $Output and then invokes my new template AddOutlookContact.tpl:

$smarty->assign("IMPORT_DETAILS", $Output);
$smarty->display("AddOutlookContact.tpl");

In the guts of AddOutlookContact.tpl, the output is reproduced:

<div>{$IMPORT_DETAILS}</div>

The tasks within AddOutlookContact.php are:

Get passed parameters
Check parameters
Find any matching names or emails
Show matches to user
Show Add Contact button

So: the user can see any existing matches, and can decide whether or not to press the Add Contact button, which goes to another script AddOutlookContact2.php.

The SQL script to query the existing contacts is as follows:

SELECT c.*, a.accountname, crm.deleted FROM vtiger_contactdetails AS c
LEFT JOIN vtiger_account AS a USING (accountid)
INNER JOIN vtiger_crmentity AS crm ON crm.crmid=c.contactid
WHERE lastname LIKE '%$QuotedLastName%' OR email LIKE '%$QuotedEmail%'";

Note that I also use the function mysql_real_escape_string() to ensure that the passed strings cope with single and double quotes etc.

The SQL links vtiger_contactdetails with the vtiger_crmentity table so the Deleted column can be picked up. The vtiger_account table is also included so any associated account name can be found.

The code goes through all the found matches and displays the results to the user. I provided a link to each contact - this is the code for the link:

$MatchLink = "index.php?action=DetailView&module=Contacts&record=$Match_ContactId&parenttab=Support";

The following form shows a suitable "Add new contact" button:

<form action='index.php' method='post' onsubmit='VtigerJS_DialogBox.block();'>
<input type='hidden' value='AddOutlookContact2' name='action' />
<input type='hidden' value='Contacts' name='module' />
<input type='hidden' value='$htmlLastName' name='LastName' />
<input type='hidden' value='$htmlFirstName' name='FirstName' />
<input type='hidden' value='$htmlEmail' name='Email' />
<input type='submit' value='Add new contact' />
</form>

AddOutlookContact2.php has a similar core as before, except it generates HTML that shows what it has done. Here's what I did to add a new contact. This is probably not the recommended vtiger method, but it seems to get the job done.

$adb->startTransaction();
$NewContactNo = $focus->setModuleSeqNumber("increment",'Contacts'); // updates vtiger_modentity_num
$focus->insertIntoCrmEntity('Contacts'); // updates vtiger_crmentity and sets $focus->id

$query = "insert into vtiger_contactdetails (contactid,contact_no,firstname,lastname,email) values(?,?,?,?,?)";
$qparams = array($focus->id, $NewContactNo, $FirstName, $LastName, $Email);
$adb->pquery($query, $qparams);

$query = "insert into vtiger_contactscf (contactid) values(?)";
$qparams = array($focus->id);
$adb->pquery($query, $qparams);

$query = "insert into vtiger_contactaddress (contactaddressid) values(?)";
$qparams = array($focus->id);
$adb->pquery($query, $qparams);

$query = "insert into vtiger_contactsubdetails (contactsubscriptionid) values(?)";
$qparams = array($focus->id);
$adb->pquery($query, $qparams);

$adb->completeTransaction();

Here are the steps:

Call setModuleSeqNumber() to get the next contact no eg CON2
Call insertIntoCrmEntity() to get the next contact id, eg 794
Insert the basic details into vtiger_contactdetails
Insert the contact id into vtiger_contactscf
Insert basic rows into vtiger_contactaddress and vtiger_contactsubdetails

The output provides a suitable link to the new Contact.

Later, I made a real Outlook 2007 addin to call this vtiger code. An extra button is shown on the ribbon when you read an email. When you click the button, it picks up the sender's name and email address and shows the correct URL with parameters in the user's default browser.

DNN no container span causes validation error

2009-08-26T15:28:00.010+01:00

If you turn off the DNN container for a module, then DNN by default inserts an extra SPAN that can cause a W3C validation error. One solution is to have a minimal container.

I had turned off the "Display Container" option for some modules so that no title is displayed. (In Admin Edit mode, the default container is used so that you can get at settings etc.) The HTML produced by DNN looked like this, within the skin pane:
<a name="402"></a>/
<span id="dnn_ctr402_ContentPane" class="DNNAlignleft">

<div id="dnn_ctr402_ModuleContent">
and then the module content.

This causes a validation error because you cannot nest a DIV within a SPAN.

My solution is not to use the "Display Container" option - instead, I created a container with no title and used that instead. In the replacement container, I moved the [ACTIONS] and [ICON] into the footer. This NoTitleContainer container files are available here: http://www.phdcc.com/download/DNN/NoTitleContainer.zip. Unzip in /Portals/_default/Containers/MinimalExtropy. I'm not sure if you need to go to [Admin][Skins] to make the container visible.

DNN Event Viewer spider errors

2009-06-25T22:26:00.005+01:00

If you are getting "Page Load Exception" errors in the DotNetNuke (DNN) [Admin][Event Viewer] with the UserAgent indicating a web crawling spider, then you need this fix:

The fix: Get the Browser Caps - v2 by Oliver Hine. Unzip the file App_Browsers.zip to get the file OtherSpiders.browser - put this in the App_Browsers directory of your DNN site. Note that your DNN site will automatically restart, so perhaps do this at a quiet time.

Alternatively (if you want), you should be able to stop these spiders accessing your site using a suitable robots.txt file.

This patch is needed for DNN 4.9.4 and possibly earlier. I think that it may be fixed in DNN 5.1+.

Spiders fixed: This file has fixes for these spiders: Baiduspider, Yandex, ia_archiver, Sphere, Feedfetcher-Google, Yanga, worio, zibber, ~~twiceler, voliabot~~, aisearchbot, robotgenius and R6_FeedFetcher.

Spiders not fixed: However it doesn't fix TweetmemeBot - I failed to add support for this. Read this Tweetmeme.

Update: doesn't seem to work for Twiceler or voilabot.

What's happening: DNN code is asking ASP.NET for details of the browser capabilities. ASP.NET gets this from the UserAgent string. DNN asks for the major and minor version numbers of the browser and causes an exception if these cannot be determined from the UserAgent. ASP.NET uses various .browser files (in the framework directories and in the web app's App_Browsers directory) to work out what browsers can do. As the UserAgent provided by these spiders doesn't follow a standard regex pattern, ASP.NET cannot get the major and minor versions.

Thanks to: Barry Sweeney for his prompt help on Twitter - and Oliver Hine of course.

Below is the DNN exception that I get before applying this fix:

Message: DotNetNuke.Services.Exceptions.PageLoadException: Value cannot be null. Parameter name: String ---> System.ArgumentNullException: Value cannot be null. Parameter name: String at System.Number.StringToNumber(String str, NumberStyles options, NumberBuffer& number, NumberFormatInfo info, Boolean parseDecimal) at System.Number.ParseInt32(String s, NumberStyles style, NumberFormatInfo info) at System.Web.Configuration.HttpCapabilitiesBase.get_MajorVersion() at DotNetNuke.UI.Utilities.ClientAPI.BrowserSupportsFunctionality(ClientFunctionality eFunctionality) at DotNetNuke.UI.Utilities.ClientAPI.get_ROW_DELIMITER() at DotNetNuke.UI.Utilities.ClientAPI.RegisterClientVariable(Page objPage, String strVar, String strValue, Boolean blnOverwrite) at DotNetNuke.UI.Skins.Controls.Search.Page_PreRender(Object sender, EventArgs e) at System.Web.UI.Control.OnPreRender(EventArgs e) at System.Web.UI.Control.PreRenderRecursiveInternal() at System.Web.UI.Control.PreRenderRecursiveInternal() at System.Web.UI.Control.PreRenderRecursiveInternal() at System.Web.UI.Control.PreRenderRecursiveInternal() at System.Web.UI.Control.PreRenderRecursiveInternal() at System.Web.UI.Control.PreRenderRecursiveInternal() at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint) --- End of inner exception stack trace ---

DNN Skin token support in a module

2009-05-27T16:41:00.007+01:00

This article shows how to get a DotNetNuke (DNN) module to support tokens in skins. It will use, as an example, how I added support for the [CODEMODULE] skin token in our phdcc.CodeModule retail module.

Introduction: skins are used by DNN as a template for the whole site output, and a container template is used to wrap an individual module. Skins and containers can contain standard tokens such as [LOGO] - this is replaced at runtime by the logo image chosen by the site administrator.

Doing it manually

The crucial trick is to add a suitable entry to the ModuleControls table.

This table primarily contains a mapping between the blank (View), Edit and Settings control keys for a module and the actual controls that implement them, eg the "Settings" ControlKey is set to "DesktopModules/phdcc.CodeModule/Settings.ascx" for my module.

To support your new skin token, add a new row with your skin token name in the ControlKey column (eg "CodeModule" with no quotes or brackets), a NULL for the ControlTitle, your view control in ControlSrc and -2 in ControlType. For my module, the ControlSrc is "DesktopModules/phdcc.CodeModule/View.ascx"

Adding a token to a skin

People work with skins in various ways. The standard method is to have an HTML template file that is parsed into an ASCX when the skin is uploaded. If you edit the HTML file on a live site then you can click on [Parse Skin Package] to re-make the ASCX.

Anyway, add your skin token to your HTML skin somewhere, eg add "[CODEMODULE]", without quotes but with brackets - and in upper case. Either package it up and upload, or edit the live version and click on [Parse Skin Package]. The parse log should show your token being replaced. The ASCX should have your token replaced, as per the example below.

If you work with an ASCX template file, then you need to add a suitable Register directive at the top of the ASCX, eg:
<%@ Register TagPrefix="dnn" TagName="CODEMODULE" Src="~/DesktopModules/phdcc.CodeModule/View.ascx" %>
then add instances of this control later on, eg:
<dnn:codemodule runat="server" id="dnnCODEMODULE" />

Hopefully your view control should now be called whenever this skin is used. If so, hurrah! Note that the PortalModuleBase TabModuleId is set to -1 so you could use this to tell when you are being called from a skin.

Token parameters

It is useful to be able to pass parameters to your module, eg so it knows what to display. The parameters are set in the skin.xml file alongside your HTML template. For my module, I wanted to add a ControlFile parameter to tell my module what to do. I added this to skin.xml, after <Objects>

<object>
  <token>[CODEMODULE]</token>
  <settings>
    <setting>
      <name>ControlFile</name>
      <value>viewSkin.ascx</value>
    </setting>
  </settings>
</object>

You can specify several settings for your token. In this case, I set the "ControlFile" setting to value "viewSkin.ascx". Make sure [CODEMODULE] is in upper case.

To support each parameter, you must add a corresponding property to your control. So, in this case, I must add a property called "ControlFile". In VB, this could be:

Private _ControlFile As String

Public Property ControlFile() As String
  Get
    Return _ControlFile
  End Get
  Set(ByVal value As String)
    _ControlFile = value
  End Set
End Property

If you upload or re-parse your skin, you should find that the parameter has now been set in the ASCX, eg:
<dnn:codemodule runat="server" id="dnnCODEMODULE" ControlFile="viewSkin.ascx" />

When your view control is now run, the ControlFile property should have been set before your Page_Load is called. You can use this to tell when you have been called from a skin - and act accordingly.

Automatic installation

Earlier on, we added the crucial row to the ModuleControls table by hand. To set this automatically during installation, you need to add an appropriate data provider. For example, this might be called 01.00.00.SqlDataProvider. Add in SQL code to delete any existing ControlKey and add the correct one, eg:
DELETE FROM {databaseOwner}{objectQualifier}ModuleControls
  WHERE ControlKey='CodeModule'
INSERT INTO {databaseOwner}{objectQualifier}ModuleControls
  (ControlKey,ControlTitle,ControlSrc,ControlType)
  VALUES ('CodeModule',NULL,'DesktopModules/phdcc.CodeModule/View.ascx',-2)

Finally, add in an entry to your Uninstall.SqlDataProvider file to remove this entry if your module is removed:
DELETE FROM {databaseOwner}{objectQualifier}ModuleControls
  WHERE ControlKey='CodeModule'

DNN development and production sites

2009-04-14T13:49:00.005+01:00

Bro John wants me to write up how he does DotNetNuke (DNN) upgrades and site copies. As a preamble, he wrote this:
=============
I would think anyone with a production site would want three copies
A - a live site
B - a transition site
C - a development site

also
D - a trash site

As most of my stuff is now in phdcc.CodeModules, I develop these until they work on C, I copy these to D to see if they still work in a different environ. I then copy them to A and test.

I would prefer to have a B which means I can shut down A and have users running on B and then swap back to A later if for example I do a DNN version upgrade.

Doing a DNN version upgrade on a live site is just asking for trouble.

If this sort of stuff is not sorted in DNN properly then I wouldn't consider it a solid environment to do anything serious in.

I am trying to be careful about keeping all database/email server specific stuff in a few files. Hard links to other pages on the site are OK if you copy the entire site correctly.

Maybe bigger players have other tricks they pull. Maybe they swap DNS pointers. However that takes time to permeate and produces horrid cahcing problems.

Content-Location HTTP header for current URL

2009-04-03T12:30:00.002+01:00

All our web sites on shared host provider Crystaltech/Newtek have just started serving an extra HTTP header Content-Location for all static web page requests - the header contains the current URL.

To see this in action, use Rex Swain's HTTP Viewer to view this page at our web site: http://www.phdcc.com/phd.html.

If you look in the received output, you will see that the Content-Location header is set to the requested URL:
Content-Location:·http://www.phdcc.com/phd.html(CR)(LF)

Normally, the Content-Location header is used to indicate when the content actually corresponds to another URL. So if you look at http://www.phdcc.com/ you will see this output:
Content-Location:·http://www.phdcc.com/default.htm(CR)(LF)
ie the web site home page is actually called default.htm.

While this is a harmless change for most users, it did in fact fool our FindinSite-MS search engine - I am in the process of releasing a new version to cope with this. Our ISP Crystaltech claims that nothing has changed. Has anyone any further information?

Lowering ASP.NET memory usage

2009-04-02T15:36:00.013+01:00

This blog post is a work in progress on how to keep ASP.NET web application memory usage low. The motivation for this is to avoid the web app being stopped for using too much memory.

Background

The Microsoft Internet Information Services (IIS) web server has administrator options to Recycle Worker Threads, ie to automatically stop a thread that runs an ASP.NET web application if a threshold is passed. Some of these thresholds are 'arbitrary', eg the default Elapsed Time of 29 hours. However the Virtual Memory and Used Memory thresholds tend to kick in as you use more memory. If a web app is 'recycled' then you get no warning - the thread is simply terminated; the web app is only restarted in response to another web request.

In addition, a web app will tend to slow down as its memory use increases.

I am working on this topic for my FindinSite-MS site search engine. This one web app both (a) does searches and (b) crawls a web site to build a 'search database' that is used by the search. The crawl/index task is done in a separate background thread - an indexing task is either started from the user interface, or run on a predefined schedule. (The app has to be alive for a scheduled index to run - so an outside process can be set up to wake up a web app if need be.)

This software currently suffers from two problems:
- Too much of the database-being-searched is kept in memory
- More significantly: The index process uses a large amount of memory.

Note that my 'search database' db (as I refer to it) is just a set of files in memory - no real database is used. This makes the search engine easier to deploy as a database is not required.

Memory heuristics

I use the System.GC.GetTotalMemory(false) call to get the current total memory usage, without forcing a garbage collection.

I don't have a precise figure for what amount of memory is too big. On our Crystaltech shared host, anything less than 10MB is good, while anything over 100MB is bad - though a recent indexing run worked with a maximum 400MB memory usage.

Redesign methodology

My initial focus was on designing a new 'search database'. This db design needs simultaneously to be searchable and buildable, ie sufficently fast and low-memory while searching - and the same while building.

The main obvious programming technique is not to keep anything in memory. This is actually quite a hard mindset to achieve. For example, it would be quite nice to keep a bit of information in memory about every file indexed, eg URL, title, size, etc. While this might work for 1,000 files, or even 25,000, it is not going to work for half a million.

Using disk instead of memory

If I cannot store information in memory, then I'll have to save it to disk. In fact, this may not be as bad an option as it sounds, as the operating system (and hardware etc) will cache disk data in memory, so performance time may not go down too significantly. Storing data on disk should not impact on ASP.NET memory usage, so should help ensure that my app isn't killed.

Example 1: word file list

I want to store a list of file numbers for each word found during the crawl. My first re-design had 32 of these numbers in memory, along with 4 other integers - with the rest on disk. A second redesign reduced this to 8+4. My latest design simply has 2 integers per word in memory. Everything else is on disk.

The first integer is the block number in the temporary data file. The second number is the last inserted file number - this makes sure that I don't update the block more than once per file for each word.

Example 2: file list

I want to know whether I've indexed a file before. I used to keep 2 (yes two) lists of files in memory. Now this is all written out to disk.

So how do I check quickly if I've already indexed a file? OK, I do have a List<> of all files indexed. Each List element is a structure that contains two integers, a FileHash and a FilePointer. The FileHash is the HashCode of the file path, and the FilePointer is the location of the full information on disk.

To check whether I've indexed a file, I find the HashCode of the file path. I then iterate through the List<>. If the hash matches then I use the FilePointer to retrieve the path from disk. If this matches, then the file has been indexed before. I keep looking if it doesn't match, in case two or more files have the same hash.

Example 3: reversed word list

To support wild cards at the start of a search, eg a search for [*hris], I need to reverse each word, so that "chris" becomes "sirhc". [I won't reveal my algorithm just now.]

At one stage, I created a list of reversed words as I went through the normal word list. However, I now write the reversed words out to disk first. I then clear the word list (see below) to reduce my memory footprint. Finally I read my reversed words in for processing.

Example 4: SortedDictionary.Clear()

I use a SortedDictionary during the indexing run. If I set this to null and do a garbage collect, then no memory is freed. If I call Clear() and then set this to null, the memory is cleared. [And I'm pretty certain that there are no other references to items in the dictionary.]

Should I use Cache?

Is it safe to use the ASP.NET Page.Cache? I presume that using this will still add to the memory used by the application, so the web app could still be shut down unilaterally, without trying to clear the Cache.

I do use the Cache as a part of the search process. However I set an expiry time of 5 minutes - this provides a useful cache while a user is searching, but clears the memory when they have probably gone anyway.

Storing data in the Session

I have just remembered that I store the search results for each user in a Session variable. This is useful, eg when they ask for the second page of hits for a search. This results data could be reasonably big, so I now think that this is not wise. The Session variable will presumably be cleared after say 20 minutes, but this is too big a risk. I'll have to store the results to disk, retrieve and clear as necessary.

Large Object Heap objects

This article on The Dangers of the Large Object Heap says that any object larger than 85kB (or 8kB for doubles) might result in increased memory usage. Try not to use large objects.
Does each collection or generic collection count as one object, or is each one a multitude of its individual components?

Current progress

A crawl of a 100,000 simple HTML files now takes 11 minutes and uses a maximum of 23MB of memory. When this db is loaded for searching, the rest state of the web app is 6MB. After a couple of searches this went up to 22MB.

The previous version took 83 minutes and used 126MB max memory. When loaded for searching, 44MB of memory is used, going up to 50MB after a couple of searches.

This is not exactly a completely fair test as the new code is not complete in several important ways. However it does show dramatic memory usage reductions. I am not exactly sure why there's such a dramatic speed improvement - it might be because of the reduced memory usage - or it could be the simpler not-really-complete algorithm.

DNN portal Import/Export Link Url bug

2009-03-29T13:40:00.007+01:00

In DNN DotNetNuke 4.92, you can use [Host][Portals] to export a complete Portal as a template to the Portals/_default/ directory. You can then import the template elsewhere using [Admin][Site Wizard].

Summary: if you have pages that have a "Link Url" that refers to another page on the site, then these links do not survive the import/export process properly. Currently, you will need to fix these up by hand after the import, going through [Admin][Pages] to edit the selected page's settings.

Preamble: in DNN, each page is identified by an integer TabId. This nomenclature stems from DNN's initial versions that refered to each page as a "tab".

Details: When a "Link Url" is set up to another page on the site, the page's TabId is stored as an integer in the Tabs database table Url column. When the portal is exported, this Url TabId number is stored in the template (in a <url> XML element). On import, the Url TabId is simply stored directly in the new page row Url column. However, the TabId of the desired "Link Url" page will almost certainly have changed, so the page will redirect the user to the wrong page.

Background: The Site Wizard does not really describe what it does very well. An old book says that the import process is *additive*, ie the pages in the template are *added* to the existing pages on your site. However this isn't entirely correct: if you select "Replace" in the wizard, then the existing pages are soft-deleted, ie their TabName has "_old" appended and the page is marked as Deleted.

The import Site Wizard assigns a new TabId to each imported page (if a new page is created). This means that it is virtually certain that the TabId-s have changed. If your template etc has hard-coded links to particular TabId-s, then these are almost guaranteed to be wrong.

It would be useful if it were possible to import a template and keep the original TabId-s. Currently, the portal export does not contain the current TabId for each page. Superficially it is also not possible to keep existing TabId-s as the Tab table TabID column is an identity. However it is possible to use the SET IDENTITY_INSERT command to insert a row with a particular TabId.

Reported on DNN Gemini bug tracker

To do:
- See if there is a better way of putting links (eg in Text/HTML and in code) so that they survive the import/export process. Me: in code you can do this using the TabController GetTabByName function.
- Double-check what the Ignore and Merge site wizard options do

Bro John adds:
- These problems typically cause an infinite redirect loop and that IE does not notice this but FF and Chrome do. Me: The problem in one case is that the Login page was not visible to anonymous users, so it redirects to the Login page ad mausoleum. The import process does not reset the LoginTabId; it only sets it if a tab with tabtype logintab is found. Unless the old LoginTabId is Replaced, this should be safe.
- The logon menu item typically fails so you need to have added a hand made login page to be able to log in to the site once you have created it ...