Save As HTML… for chat logs

[neil_mayhew]

It would be very helpful to be able to save logs as HTML for sending to another person who wasn't in the chat (or even someone who was).

XSLT could be used to transform the .chatlog files to HTML. Attached is an XSLT stylesheet that I have written to do this. The libxslt library is present on all versions of Mac OS X since 10.2, and its command-line tool can be invoked in a sub-shell using system(3):

xsltproc -o 'foobar (2007-03-08T09.19.11-0700).html' format-html.xsl \
            'foobar (2007-03-08T09.19.11-0700).chatlog'

[cbarrett]

I'll look this one over and add it to the tools directory assuming it works okay after testing.

In the meantime, Neil, can you put license preludes on these files? GPL or BSD/MIT, whichever you prefer. A diff adding your name to Copyright.txt would be swell, too.

[chimerical]

This is mostly related, but could we also have the option to have logs generated in HTML format as the previous versions of Adium had done? The current XML format is great, and I'm not saying we scrap it. Since the output to HTML was already in existence in Adium before, I figure it should be easier to put it back into Adium as a user option.

[cbarrett]

Actually that's an incorrect assumption, chimerical. Why do you want the old HTML style logs?

Evan, we can't commit these files without some sort of licensing. Right now they have none. :(

[evands]

Replying to cbarrett:

Evan, we can't commit these files without some sort of licensing. Right now they have none. :(

Files posted publically without a license are accepted to be in the public domain.

[chimerical]

Replying to cbarrett:

Actually that's an incorrect assumption, chimerical. Why do you want the old HTML style logs?

Coming from years of HTML logs by AIM Windows + DeadAIM, the HTML format logs have been portable in that the files can be viewed on any computer with a browser (even if Adium's transcript viewer is not available). Also, I assume HTML file format logs can be searched by Spotlight and Google Desktop.

[edr1084]

Replying to chimerical:

Also, I assume HTML file format logs can be searched by Spotlight and Google Desktop.

Our new logging format is also spotlight searchable.

[boredzo]

Replying to evands:

Files posted publically without a license are accepted to be in the public domain.

Works created and published without an explicit waiver of copyright are still copyrighted; they are not public domain. Treating them as such is a good way to get sued someday. (Disclaimer: IANAL.)

Colin is right: We need the patch author to add an explicit license or waiver of copyright to these files.

[Robby]

We need feedback here.

[edr1084]

I have contacted the user directly so I'm resetting the pending flag. If I don't get a response we can close this one and I (or anyone else who decides to) can rewrite this.

[Robby]

Ok, no problem. :)

[iapx]

It would be a cool idea to include link of received file, or even includes <IMG/> tags for received/sent pictures.

[neil_mayhew]

Many apologies for the long delay.

I've added some improvements to both the shell script and the XSLT stylesheet. Most importantly, as requested, I've also added license blocks and added my name to the Copyright.txt file. I copied the text of the block out of one of the Adium source files. To avoid any confusion, I took my name out of my source files.

I'm uploading the two new files, and a patch for the copyright file.

My original intention was that the Save As... menu item would be enabled in the Log Viewer, and you would be able to choose an output file name and directory into which the HTML would be saved. The code that handles the menu item would simply shell out to my script, which computes a title and runs the stylesheet. I'll look into implementing this in Adium and submit a patch, if possible.

Ideally, any conversion to HTML would use the user's current theme, or one selected from a list, but it would take a good bit more work to figure out how to do this. Maybe the code that generates the HTML content of the message window could be adapted to do this instead? However, I'm not sure that the logged XML contains all the information of the original.

I often end up sending chatlogs of meetings to friends and colleagues, and they all say they appreciate the easy-to-read HTML formatting that I generate. So I think that a built-in facility to do this would be appreciated.

[neil_mayhew]

Shell script for batch conversion of .chatlogs to .html using XSLT (updated version)

[neil_mayhew]

Patch to add name to Adium copyright file

[neil_mayhew]

Sample chatlog, based on Message Preview pane from Preferences dialog

[neil_mayhew]

Sample HTML conversion, based on Message Preview pane from Preferences dialog

[Robby]

This needs to be looked at.

[evands]

Let's not add new features during the beta period; agreed that this longstanding ticket should be looked at sooner than later :)

[Robby]

At least the ticket got your attention, heh. ;)

[jmpp]

Three comments on this enhancement request from a user who has been wanting the same functionality for a while now, namely me ;-)

1. Maybe one easy way to select the HTML theme for the exported chatlog would be a combination of the OP's two suggestions: default to the currently selected theme (Adium's preferences --> Messages --> Message Style) in a pop-up menu --located in the "Save As..." dialog when you're exporting a chat transcript-- that offers you the ability to choose a style from any of the currently installed themes.

2. Maybe additional XSLT stylesheets could provide the additional functionality of exporting to other file formats, like plain text, rtf, xhtml, etc. These could also be chosen as options from a pop-up menu in the "Save as..." dialog, for instance as "Export to: (HTML | RTF | Plain Text | etc.)", with a suitable default (html?).

3. Lastly, a third available choice could be file encoding, just in case, defaulting of course to UTF-8. Mac OS X comes standard with libiconv, so hopefully this wouldn't be that complicated.

Regards,...

-jmpp

PS: I'll provide patches for these things once I finally learn Cocoa, if no one beats me to it ;-)

[Alex]

This post may be a bit late but I have converted one line of the XML chat logs (.chatlog) to a text readable format with applescript. This is a sample: From: <message sender=\"sender1@…\" time=\"2008-07-23T13:15:15-03:00\" alias=\"Alex\"><div><span style=\"font-family: Verdana; font-size: 11pt;\">Hi Everyone</span></div></message>

To: Alex (13:15:15): Hi everyone

I realize that the time is in 24-hour format and some other bugs. I will post the script when I figure out how to "mass-convert" the lines and fix some bugs.

[neil_mayhew]

Replying to Alex:

I have converted one line of the XML chat logs (.chatlog) to a text readable format with applescript.

Alex, I'm not sure what this adds, other than using AppleScript instead of XSLT. The original scripts I posted here do all this and a lot more. We are just waiting for a dev review.

[neil_mayhew]

However, we do need a way to invoke the Save As HTML operation from the GUI, and maybe using the script menu would be one way to do it. There's a do shell script verb in AppleScript that could be used to invoke the shell script, which in turn invokes the XSLT. The only thing that's needed is the full path of the log file, but I don't think it's possible to get this via AppleScript. If it was, all kinds of log transformations would be possible without having to change the Adium code itself.

I looked at adding a Save As menu command to the log viewer but, not being a Cocoa programmer, I couldn't see how to do it. I suspect that because the Log Viewer is implemented as a dialog, it doesn't have a connection to the menu bar. If so, it would need a button on the dialog somewhere, and this could start to clutter the interface.

[evands]

• Connect the Save As menu item in Adium's MainMenu.nib to the First Responder's saveDocumentAs: action. Any window in Adium can then respnd to saveDocumentAs: to do its saving.
• Implement - (IBAction)saveDocumentAs:(id)sender in the log viewer's window controller. This should display an NSSavePanel prompt for the destination file, then do the actual saving.
• Bonus points: Implement saveDocumentAs: for message windows, too, allowing the log to be saved from the message window itself.

[neil_mayhew]

Cool! Thanks for the pointers, Evan. It sounds like I would be able to do that, although not for another few weeks as I am on a trip at the moment and don't have my Mac with me. I'll have a go when I get back.

[evands]

Excellent :) Let me know if you run into any problems when you do tackle it. Cocoa is easy when you already know what you're looking for but can be quite confusing if you don't; I'm happy to help, so don't be afraid to ask.

[cbarrett]

This is pretty big for just a bugfix release.

[zacw]

I've been thinking about doing this, though recently-included logged images is one of the reasons against how I wanted.

We could take the HTML source of the log view window and save it. The downside is we have to encode all of the images  using a data: URI. This has the upside of being incredibly portable, but the downside of being 33% larger. A flat file which could be sent/etc would be very useful.

[assetburned]

hmm 3 years and counting... this is a good idea. wondering why it isn't implemented already?

[Robby]

Replying to assetburned:

hmm 3 years and counting... this is a good idea. wondering why it isn't implemented already?

Because you haven't provided a patch for it, yet. :)

[assetburned]

right, but I'm working on another Adium related project at the moment. but this ticket was a big help for me. :-)

could someone update the format-html.xsl for the log new 1.4b15 format? I'm getting errors for the new kind of logs. I have a version but the output is horrible.

would also be interesting to know how to deselect some parts of the output e.g. the sender-ID or the alias could be removed from the message lines in the HTML output.

[neil_mayhew]

Replying to assetburned:

this ticket was a big help for me. :-)

Glad to hear that :-)

could someone update the format-html.xsl for the log new 1.4b15 format?

I think the difference is the presence of a new alias attribute on messages and statuses. This is actually very helpful, and it wasn't too hard to update the stylesheet to use it.

I'm getting errors for the new kind of logs.

I assume you were getting "Unhandled attribute" messages on stderr.

I have a version but the output is horrible.

Is this because the error messages were getting interspersed with the HTML? Or what were you seeing exactly?

would also be interesting to know how to deselect some parts of the output e.g. the sender-ID or the alias could be removed from the message lines in the HTML output.

I changed the stylesheet so that it uses the alias, if present, and the sender otherwise. Let me know if this isn't helping. You should be able to see how I did it, if you compare the old and new stylesheets. However, feel free to ask more if you need to.

I've uploaded a new stylesheet, and also a DTD for the chatlogs.

I hope I'll eventually be able to provide some integration into Adium itself, but my Mac broke a year or so ago and I've only just gotten a replacement.

[neil_mayhew]

XSLT stylesheet for converting chatlog to html

[neil_mayhew]

DTD for chatlog files

[assetburned]

Hi,

Sorry to hear that with your Mac :-( Hm, I'm pretty new to this xsltproc stuff. What is the DDT good for?!

Anyway, the problem was indeed related to the Alias tag. But that wasn't everything. The new chatlog also logs status messages such as away or coming back again. And those messages are not shown if I apply your XSL to the log.

I added an example log file.

By the way. I don't know if it is me or my MacBook or what, but some of the colours you use for the background are very hard to see. I replaced F6FFED by BBEE99 and F7FAFF by CCDDFF.

cu assetburned

[assetburned]

hm crap next time i know it better. i can't upload a file to this topic :-/

So check out pastebin for the log file :-/  http://pastebin.com/f7a0b795c

[assetburned]

example log of 1.4b15

[neil_mayhew]

Replying to assetburned:

What is the DDT good for?!

Actually, it's a DTD, which stands for "Document Type Definition". It's a definition of the allowed structure ("syntax", if you like) of the XML file. This can be used by various external tools to validate the XML file against the expected structure. xmllint and xsltproc will both use the DTD if it's available. This helps to ensure that the stylesheet is not getting input it wasn't designed for. I just added the alias attribute to it, since this is a change to the XML structure that Adium is generating.

The new chatlog also logs status messages such as away or coming back again. And those messages are not shown if I apply your XSL to the log.

Yes, I see what you mean. I'll try to fix this.

I added an example log file.

Thanks -- very helpful.

By the way. I don't know if it is me or my MacBook or what, but some of the colours you use for the background are very hard to see. I replaced F6FFED by BBEE99 and F7FAFF by CCDDFF.

I wanted the backgrounds to be very pale and subtle. Maybe a bit too subtle! Eventually I'd like to add a parameter to the stylesheet that lets the colors, or at least the saturation, be specified externally. However, for now, I'll focus on the essentials.

[assetburned]

Ah ok. So I have to do a little more read up work about it.

Anyway I think I just started another little trouble maker, with your help :-)  http://www.adiumxtras.com/index.php?a=xtras&xtra_id=7161 just to let you know what your work could be used for.

[Robby]

We'd like to have this in 1.5, neil_mayhew. Are you going to update your patch? :)

[Robby]

Ticket #14339 has been marked as a duplicate of this ticket.

[purpleuser]

Trying Pidgin; the XSL file works, but isn't Pidgin native (Pidgin does not understand CSS). I don't know XSL to fix that. If you're working with Pidgin,  see here.

In the xsl file,  http://purl.org/net/ulf/ns/0.4-02 ( http://soc.hbar.us) is dead, replaced by XMLLogFormat

I've learned about xmllogformat. Excellent format, and all chat programs should implement it. I feel bad about stripping the markup for Pidgin. In the meantime, Adium needs an export feature to help make it standard. I do see a deactivated "Save as" in the 1.5 nightly. Cheers

[Iritscen]

Looking forward to this getting added. In my experience, Spotlight does not index .chatlogs, so for all the old logs which I moved out of Adium's Logs folder, the only way to get them indexed by the OS is first to convert them to an ordinary format. This shell script saved me the trouble of having to write my own batch-conversion script, so thank you (although I was going to go to RTF because it has far less markup overhead, and it's faster to open TextEdit than a browser to view a file ;-).

The one issue I noticed is that the script does not handle spaces, changing them to "-"s and stripping non-alphanumeric characters like "(", which ends up creating a new path on the hard drive using the mangled folder names rather than placing the converted file besides the original. Probably not important, in that it will get fixed when it's implemented in Adium.

Now, can Adium allow batch conversion as part of this forthcoming log export menu option? It seems to me that users are at least as likely to need mass log exporting and conversion as they are to need this for a single file. But perhaps that's just my own particular user story. Sorry if this has been discussed elsewhere, I haven't been keeping up with Adium's Trac lately.

[Iritscen]

Stylesheet for RTF exports

[Iritscen]

Hello again. I decided I needed to modify the script for my own needs after all, so I figured I might as well post what I ended up with, rather than keeping it to myself. I didn't understand some of the design choices made in the original script, so I didn't carry them over, but my version seems to be functionally the same, and no longer mangles file paths.

At the very least, "export_adium_chatlogs.sh" might be useful to people who want to run it directly in Terminal to convert old chat logs. I'm actually not clear on whether the devs were seriously thinking of incorporating a shell script directly into Adium.app or not, but it should work nicely with Adium too, if they wish (note to devs, the script returns specific exit status codes for errors).

Main changes from "format-html.sh":

• Can export a folder of chat logs or a single one.
• Supports different stylesheets, but you must now name them "format-suffix.xsl" and pass them in explicitly with "-s format-xxxxx.xsl" (suffixes can be up to 5 characters).
• Requires the chatlog.dtd file to be present now, as a safeguard.
• Three choices for export method:
• 1. Put exported logs next to originals (original behavior from "format-html.sh")
• 2. Put all exports in one folder together (simplest option)
• 3. Copy the hierarchy of the original folder structure (useful for exporting Adium's Logs folder with its subfolders by IM account preserved).

Consult the script's built-in help for details. I ran extensive tests and used it to convert my 50MB of old logs*, but please let me know if you find any issues, as this is still a 1.0 release and therefore bugs are inevitable.

(*Actually, there were five failed exports out of the 2300+ logs, which seem to stem from certain strings in IMs causing problems for the XSLT parser. Perhaps we can get to the bottom of that another time -- I can provide the strings that caused the errors.)

I'm also providing a rudimentary stylesheet for going to RTF, for those who find it an easier and more aesthetic format to read chats in. An RTF conversion weighs in at about half the size of the original .chatlog and a quarter the size of an HTML conversion. My goal was to replicate the simple appearance of the chat log when viewed by Adium's Transcript Viewer.

Thanks to neil_mayhew for introducng me to xsltproc; I never knew we had such a handy program included on our command line! Because I am still new to XSLT, I was unable to correct the bug in "format-html.xsl" in converting the "You have disconnected" messages, so "format-rtf.xsl" has the same issue. Hopefully this is easy for someone else to patch.

[Iritscen]

New version of format-html.sh

[Iritscen]

Corrected a documentation typo on line 225 of the script ("non-mirrored"->"flat-folder").

I do believe that a log-exporting feature is long overdue in Adium, especially because Spotlight does not index .chatlogs and also because you cannot send the .chatlog to someone without Adium to read (even if they have Adium, double-clicking the .chatlog only seems to open the Transcript Viewer that shows what is in Adium's Logs folder, rather than actually opening the document). Let me know if there is anything I can do to make this feature a reality in Adium.