Visualising Geotagged Content with Google Maps

Following Steve Reiner’s Twitter post last week, I was inspired over the weekend to add a similar Google Maps-based dashlet to share-extras, to show the locations of geotagged content items on a map view.
Since the repository has full support for extracting geographic data using Tika in version 3.4, all I needed to do to assemble some test content was upload a few photos taken on my phone into the site Document Library.
If you look at the Document Details page of a geotagged photo, you’ll see that this displays a latitude and longitude value at the end of the item’s properties list. These are part of a new aspect named Geographic.

Latitude and Longitude on the Document Details page

Latitude and Longitude


Using Firebug’s Net console, I noticed that the JSON data for the document list view makes these values available on a new geographic property placed on each list item.
Firebug Net Console

Firebug's Net Console


So to keep things simple the initial version of the dashlet simply re-used the doclist web script to grab a list of all content items in the root of the document library space, but the final version now on share-extras comes bundled with a dedicated webscript to list all items in the site that have the Geographic aspect applied.
Using this data, the dashlet displays a marker for each geotagged item, auto-centering itself on the centre point of all the items.

Clicking on a marker takes you to the Document Details page for that item. In the next update I’ll look at displaying a snippet of information for the item, which the Google Maps API makes pretty easy.

From Web Scripts to Portlets with Alfresco 3.4

Since Alfresco 3.2r the ProxyPortlet support in Alfresco Share has allowed developers to easily embed specific bits of Share functionality into a portal such as Liferay, using only a small amount of XML configuration to wire in existing web scripts.
This support has been substantially improved in version 3.4 of Alfresco, in order to allow the entire Share Document Library page to be embedded within a portal. Unfortunately the changes mean that the steps in Luis’s original tutorial no longer work in the latest version.
As one of the features we demonstrated today at our Madrid event was the Doclib portlet, I managed to get five minutes to get the original web script-backed method working too.
Since the CMIS repository browsing web scripts used in Luis’s example are no longer shipped with Share, I used my own Hello World dashlet from share-extras as a starting point instead. The web script is basic, but demonstrates displaying a simple greeting to the user including their user name.
Hello World Portlet
The following steps should work using a recent version of Alfresco 3.4 and Liferay 5.2.3 running as the portal, provided that the two components are first set up as per the Installing and Configuring instructions for the Doclib Portlet.
Once you’ve set everything up, the first thing to do is to add the web script files to the instance of Share that you have already deployed to Liferay. Since you should already have created some directories in Liferay’s tomcat-x.x.x/shared/classes directory to define your share-config-custom.xml, the easiest thing is to create a new directory named site-webscripts within the existing tomcat-x.x.x/shared/classes/alfresco/web-extension and place the following files in it.
org/alfresco/test/hello-world.get.desc.xml

<webscript>
   <shortname>Hello World</shortname>
   <description>Displays Hello World text to the user</description>
   <authentication>none</authentication>
   <url>/test/hello-world</url>
</webscript>

org/alfresco/test/hello-world.get.html.ftl

<html>
   <head>
      <title>${msg("header")}</title>
   </head>
   <body>
      ${msg("label.hello", user.id)}
   </body>
</html>

org/alfresco/test/hello-world.get.properties

header=Hello World!
label.hello=Hello {0}

With those files added, you’ve successfully defined the web script that we’ll wire into Share in the next section.
Now although the web script itself will be automatically picked up by Share at load-time, some additional config is also needed in web.xml for the ProxyPortlet to work in version 3.4.
The following lines, which define a custom servlet and servlet mapping which will be invoked by the ProxyPortlet, should be placed in the web.xml file belonging to the Share instance which has been deployed in Liferay. You should find the path to this will be something like <LIFERAY_HOME>/tomcat-6.0.18/webapps/share/WEB-INF web.xml (if you have not already started Liferay you will need to do so to force it to deploy share.war and create this structure).

   <servlet>
      <servlet-name>HelloWorld</servlet-name>
      <servlet-class>com.liferay.portal.kernel.servlet.PortletServlet</servlet-class>
      <init-param>
         <param-name>portlet-class</param-name>
         <param-value>org.alfresco.web.portlet.ProxyPortlet</param-value>
      </init-param>
      <load-on-startup>0</load-on-startup>
   </servlet>
   <servlet-mapping>
      <servlet-name>HelloWorld</servlet-name>
      <url-pattern>/HelloWorld/*</url-pattern>
   </servlet-mapping>

You can place these two definitions anywhere within the top-level <web-app> element, but for consistency I always try to add them next to the existing <servlet> and <servlet-mapping> definitions.
Now you’ve done all you need to do to configure the scripts, we can move onto configuring the matching portlet definition which will be picked up by Liferay.
In the same WEB-INF directory where you modified web.xml you should find a file named portlet.xml, to which we add our new definition.

   <portlet>
      <description>Hello World</description>
      <portlet-name>HelloWorld</portlet-name>
      <portlet-class>org.alfresco.web.portlet.ProxyPortlet</portlet-class>
      <init-param>
         <name>viewScriptUrl</name>
         <value>/page/test/hello-world</value>
      </init-param>
      <supports>
         <mime-type>text/html</mime-type>
         <portlet-mode>VIEW</portlet-mode>
      </supports>
      <portlet-info>
         <title>Hello World</title>
         <short-title>Hello World</short-title>
      </portlet-info>
      <security-role-ref>
         <role-name>administrator</role-name>
      </security-role-ref>
      <security-role-ref>
         <role-name>guest</role-name>
      </security-role-ref>
      <security-role-ref>
         <role-name>power-user</role-name>
      </security-role-ref>
      <security-role-ref>
         <role-name>user</role-name>
      </security-role-ref>
   </portlet>

Add this right after the existing <portlet> definitions (which if you look at further it should be obvious define the three Doclib portlets) and save your changes.
Getting these details right is crucial if you’re deploying your own web scripts, so a couple of notes on this are probably useful.

  1. The contents of the <portlet-name> element must match the name of the servlet mapping you have defined in web.xml
  2. The viewScriptUrl parameter must match the URL of your web script, with a prefix of /page added to the beginning (note that in 3.2r the web app context path was also required in the URL, but this now causes an error if supplied)

Lastly you should add the portlet to the two Liferay-specific configuration files in WEB-INF to ensure that authentication is handled correctly and also that the portal appears in the correct category in Liferay’s Applications menu.
In liferay-portlet.xml add the following definition after the existing <portlet> elements

   <portlet>
      <portlet-name>HelloWorld</portlet-name>
      <user-principal-strategy>screenName</user-principal-strategy>
   </portlet>

In liferay-display.xml, add the following within the existing <category> element – it should be obvious that this is adding your portlet to the ‘Alfresco’ category.

<portlet id="HelloWorld"></portlet>

You’ll need to restart Liferay to get it to pick up the new portlet, and for Share in turn to load the additional web script. Once it’s finished loading you should be able to follow the configuration steps in the Doclib Portlet guide to walk you through adding it to a page.
The example is basic, but shows how you can add a web script as a portlet, with a small amount of personalisation based on the identify of the user.
It’s possible to add more complex web scripts, for example to load data from the Alfresco repository or other back-end data sources, but as Luis points out you should be careful how you render any hyperlinks within your scripts, to ensure that they are portal-safe.

To make sure that your URLs are correctly generated, please use the “scripturl()” function in your Freemarker templates to wrap them:
<a href=”${scripturl(url.serviceContext + “/sample/cmis/repo”, false)}”>CMIS Repository</a>

You can download the web script files used in this example in JAR format, which you can extract using any unzip program into the directories specified above (or even easier – simply drop the JAR file itself into Liferay’s tomcat-x.x.x/shared/lib folder).

Share Import-Export 1.1 released

The v1.1.1 release of Share Import-Export has been up for a few days now, but I wanted to summarise some of the changes in the new version.
The most significant addition is support for importing and exporting security groups in JSON format, via the new import-groups.py and export-groups.py scripts.
As well as this the import-users.py script has been made slightly more flexible, with the addition of a --users argument to allow you to import just a few users from a larger set. Since the sample data contains a large number of users that are used across all the different sample sites, you can now import just the users you need for a particular site.
As well as these functional improvements I’ve started cleaning up the code internally, an area I intend to focus a little more on over the next few weeks. For now I’ve just cleaned up the docstrings within each script, and updating the --help flags to re-use the usage information in there.
Last but not least, I should thank Dick from Formktek for reporting an issue with the user export script, which was causing some exported profile images to become corrupted when saved.
Beyond a few more tidy-ups the code is almost where I want it to be within the current constraints of the repository. But there have been a couple of ideas suggested for future uses of the scripts, so if there’s a particular purpose you think the scripts could have or you just want to share your experiences, please leave a comment below.

Importing and Exporting Share sites

I managed to get a bit of time on my flight to Lisbon the other day to make a few last changes to a tool I’ve been working on on and off for the last few weeks, to provide a standard way of loading (and dumping) the contents of Share sites for demonstration content.
I’d considered ways of doing this in the past, mostly when I’ve got frustrated with setting up sample content in the new Share instances I usually create for demos. Bootstrapping content via ACP files is the usual method of loading content that’s employed by Alfresco’s own patches, but that didn’t really feel flexible enough and in any case, isn’t able to package up some of the data which makes up a Share site, such as configuration stored in the sitestore AVM store.
I also wanted a tool which allowed content to be imported into an Alfresco instance running on a local machine, but also on remote machines as well. Sometimes I demo locally but other times I fire up one of my Amazon images instead. So it seemed like an external process was required.
Then we started discussing a few weeks ago what additional tutorials and sample content we could provide to partners to coincide with the upcoming Enterprise 3.4 release, and the possibility of reusing some of the great demo content from the Alfresco Cloud Trial. There seemed some overlap.
The result is this project, which I’m tentatively naming Share Import-Export, and which provides a set of Python scripts that can export site-based content and user information from Alfresco Share, and import that content into another Alfresco Share server. Some sample content is also provided from the Cloud Trial environment, kindly supplied by Paul Hampton.

The package is intended to help set up demonstration environments quickly, reliably and consistently. The scripts are not suitable for importing or exporting large volumes of content or for any general production use such as system backups.
The scripts work with versions 3.3 and 3.4 of Alfresco, and can work against 3.2 with a couple of small tweaks. No additional customisations are needed in Alfresco or Share, so you can export data from existing systems without any fuss.
What can be imported/exported?
For sites

  • Site configurations
  • Site members (users only at present)
  • Site dashboards, including dashlet configuration
  • All content held within the site (manual export via ACP)
  • Records Management sites (must have RM installed)
  • Web Quick Start sites (must have WQS installed)

For users

  • All profile information, including profile images
  • User dashboard configurations
  • User preferences
  • User groups and group memberships

What is not imported/exported?

  • Document categories and tags (not currently supported by ACP format)
  • User passwords and account enabled flags (all accounts enabled)
  • Activity feed entries
  • File system-level customisations (e.g. custom dashlets) and configuration

How does it work?
The scripts mimic a web browser logging into the Share web application, then invoke a number of mostly JSON-based web scripts to read and write data to and from the application. JSON is also used as the format for storing exported metadata and user information, since it is well-structured, human readable and lightweight. Python has strong support for JSON data via the json module. ACP format is used to package up site content.
Download and Usage
Downloads and more information can now be found on the Share Import-Export project page.
Contributing
You can help by testing the scripts against your own test data and posting feedback, either by adding a basic comment below or (even better) by posting feedback in the Issues section.
You are welcome to redistribute the tools under an Apache 2.0 license and encouraged to submit any enhancements you make.

Net Neutrality

I was horrified enough at Ed Vaisey’s terrible sentiments he expressed over Net Neutrality last week, to write to my MP on the issue. Hopefully Angie will be more responsive to letters from constituents than my last MP was. Still waiting for a reply on that one…

The letter’s based on Open Rights Group’s template, but I added my own Tory-friendly additions in bold. Sending a generic letter is better than none at all, but given you’re writing to an individual it’s clearly better to tailor the argument for them.

I’d strongly encourage you to do the same if you care about universal access to information.

Dear Ms Bray,

I am writing to ask you to sign the Net Neutrality EDM 1036 first signed by Tom Watson MP, Julian Huppert MP and Peter Bottomley MP.

http://edmi.parliament.uk/EDMi/EDMDetails.aspx?EDMID=42025

Today the Coalition Government has taken a huge step towards increasing the transparency of Government by announcing the release of all central government spending data over £25,000 for the first time. You may have seen that the Prime Minister has stressed his support for this drive via a video posted this morning on the Number 10 web site.

This is a significant move which will help reduce the waste inherited from Labour and help drive the growth of an information industry which Francis Maude estimates could contribute up to £6bn to the UK economy. The work which his department has done over the last six months is making the UK a world leader in this field.

Last week however, Ed Vaizey, the Minister for Culture, Communications and Creative Industries, stated that the UK will allow Internet Service Providers to decide which websites and services can reach their customers at what speed.

This threatens the idea of free access to information to all. If traffic from established media operators is prioritised above others then this threatens the ability of independent organisations to help government find where inefficiencies exist in the system, using open data. It promotes centralism over localism and diversity in our information instructure and is a backwards step in Britain’s development.

The change – often called removing “net neutrality” or introducing ”network discrimination” has already led to complaints from companies including the BBC and Skype, an Internet telephone company, that their content may be slowed down by Internet Services Providers. ISPs, including BT, Sky and Virgin, provide TV and phone services which would give them a reason slowing down certain Internet services provided by competitors.

The danger is that, while some “traffic management” to prevent congestion may be reasonable, allowing ISPs to do what they want, with no checks other “transparency” to customers, will lead to significant market abuse and loss of innovation on the Internet. New services may not start up if they cannot be guaranteed fair access to UK Internet customers.

There are ways this problem could be prevented. One would be an industry agreement by major ISPs not to discriminate against competitors, such as has been put in place in Norway. Another would be to require “minimum service guarantees” including an Open Internet.

Please sign the EDM, and raise this issue with Ed Vaizey, as the Minister responsible.

Thanks,
Will Abson

Demo Tomcat Packages for Alfresco 3.4

Downloads and more information can now be found on the dedicated Tomcat Bundles page.
The eagle-eyed amongst you will have noticed that starting with Alfresco 3.4 a number of changes have been made to the way in which Alfresco binaries are packaged for distribution.
The new Bitrock installer makes installing the server much less challenging for new users and offers more control for power users. Also welcome should be the general tidying up that has gone on, with most packages now standardised as ZIP files, hopefully ending the which-file-should-I-download confusion.
The only casualty has been the Tomcat bundle packages, which are no longer shipped due to the overhead in maintaining these separately from the Bitrock stack and WAR bundles.
Now while they’re both great for experimental users and system admins respectively, it doesn’t beat the sheer speed of unzipping a ZIP file and double-clicking alf_start.bat, the method I’ve always used to quickly spin up demo or test instances on my Windows system. So, I packaged up my own unofficial Tomcat demo bundle for 3.4.
From 3.4.b I’ve added a Linux bundle based on the same components, but with my own alfresco.sh script that can be used to stop and start the server. It should work exactly the same as the old Alfresco Tomcat .tar.gz bundles, but with a dedicated MySQL also included.
Downloads and more information can now be found on the dedicated Tomcat Bundles page.

Adding dates in alfresco.log

By default, Alfresco’s log4j configuration specifies that only the current system time, and no date, are output as the first entry in log messages output in alfresco.log, producing output such as the following.
12:21:23,658 INFO [org.alfresco.config.JndiPropertiesFactoryBean] Loading properties file from URL [file:/home/wabson/Downloads/alfresco-enterprise-tomcat-3.3.1/tomcat/shared/classes/alfresco-global.properties]
12:21:24,009 INFO [org.alfresco.config.JndiPropertyPlaceholderConfigurer] Loading properties file from class path resource [alfresco/alfresco-shared.properties]
12:22:40,726 INFO [org.springframework.extensions.webscripts.TemplateProcessorRegistry] Registered template processor Repository Template Processor for extension ftl
12:22:40,747 INFO [org.springframework.extensions.webscripts.ScriptProcessorRegistry] Registered script processor Repository Script Processor for extension js
12:23:09,470 INFO [org.alfresco.repo.domain.schema.SchemaBootstrap] Schema managed by database dialect org.hibernate.dialect.MySQLInnoDBDialect.
12:23:11,774 INFO [org.alfresco.repo.domain.schema.SchemaBootstrap] No changes were made to the schema.

This is the line responsible for setting the configuration in log4j.properties (found in the root of the classpath in the alfresco webapp)
log4j.appender.File.layout.ConversionPattern=%d{ABSOLUTE} %-5p [%c] %m%n
That’s fine in theory, since the configuration also specifies that the log file should roll over using a date-based suffix on a nightly basis. However, in practice the log rotation can sometimes fail on running systems, leaving log entries spanning more than one day in a single log file and ambiguity over which day a particular message was output on.
Fortunately, log4j’s PatternLayout is quite flexible, and allows you to specify a date-time format instead (in fact, this is the default if no date format specifier is given.
To print the date in ISO-8601 format, e.g. “1999-11-27 15:49:37,459”, use
log4j.appender.File.layout.ConversionPattern=%d{ISO8601} %-5p [%c] %m%n
For classic date format, e.g. “06 Nov 1994 15:49:37,459”, use
log4j.appender.File.layout.ConversionPattern=%d{DATE} %-5p [%c] %m%n
To also add dates into the console logging output (which is redirected into Tomcat’s stdout logs when running as a service), update the property log4j.appender.Console.layout.ConversionPattern.

Refreshing Site Tags in Share

Sometimes the site tag data used in Share’s document library can get out of sync with the tags on the content itself, especially in older versions of the 3.x product.
I came across this today on the Partner Sales Enablement site that we’ve just launched fully today on partners.alfresco.com, but fortunately the tag scope data is easy to rebuild.
To do this, you need to create a small JavaScript file that will execute the ‘refresh-tagscope’ action against a space.

var refresh = actions.create("refresh-tagscope");
refresh.execute(space);

Once you’ve created this file (I called mine refresh-tagscope.js) in Data Dictionary > Scripts, use Alfresco Explorer to navigate into the documentLibrary space within the site you want to refresh, and click View Details from the More Actions menu.
You can then use the Run Action dialogue to execute the JavaScript file you’ve just created. You might also want to do the same for the site space itself, which also collects tag scope data.
Once that’s done you should be able to navigate back into the site document library in Share and you should find that the tag data has been fully refreshed!

Site Blog Dashlet for Alfresco Share

Updated December 2010: Site Blog Dashlet is now hosted on Share Extras.
This extension to Alfresco Share provides a custom Site Dashlet, which displays the most recent ten posts from the site blog component.

Site Blog Dashlet

Site Blog Dashlet


To install the dashlet download the ZIP file package the latest blog-dashlet.jar file from the Share Extras project and extract the contents into the tomcat directory of your Alfresco installation drop it into tomcat/shared/lib or WEB-INF/lib inside the Share webapp. The package will install the dashlet web script and a single CSS file.
Also pictured in the screenshot is the Site Tags Dashlet, which displays a tag cloud visualisation for all the tags within a site.

Seven Days to Stop the Bill

My second letter, with hyperlinks. Please feel free to use this as a template to contact your own MPs.

Dear Mr Sharma,

I wrote to you recently outlining my concerns around the Digital Economy Bill currently before parliament.

You may be aware that Harriet Harman MP announced last week that despite widespread criticism of some parts of this bill from across the creative and technology sectors, it will receive it’s second reading on Tuesday April 6, leaving only 90 minutes for the bill to be scrutinised by the House of Commons.

I hope you will agree that this is not acceptable for a Bill that seeks to define the technological landscape of Britain for the next generation. The Bill has undergone considerable scrutiny in the House of Lords and it is only reasonable to expect this same scrutiny from our elected representatives in the Commons.

I see from your web site that you have welcomed Ms Harman to the constituency on more than one occasion. I would therefore ask you to use your influence as the member for Ealing Southall to oppose the Government’s plans to rush through this Bill in the period before the election and that ensure the provisions receive proper debate and scrutiny in a new Parliament.

I am writing as one of 17,000 people who have also written to their representatives on this matter. I am sure you will have received many letters on this subject and I would ask you to take these views into account and make these known with ministers and party managers.

Yours Sincerely,
Will Abson