Parsing the WoW Armory without XML
A month or so ago Blizzard moved the WoW Armory to Battle.net servers. Currently, the new WoW Armory does not offer XML feeds for the data. I spent a few hours working with PHP and DOM to create a 'parser' for the new Armory. The below script is a trimmed down version of what is currently being used for the We Know Roster. I am only providing the back end script that will do the parsing and store the information in a MySQL database. Front end displaying can easily be achieved by querying the database with the stored results.
The script will pull the following information for each member in a specified guild: Name, Level, Class, Rank, Achievement Points, Profession 1 Name+Level, Profession 2 Name+Level, Talent1, and Talent2.
The scripts below require modifications to work properly. I recommend having knowledge of PHP/CLI before working with this script. I will develop a more user friendly version of this script only if Blizzard does not supply useful XML or JSON feeds in a reasonable amount of time.
The Bash Script:
The bash script pulls the newest HTML Roster file from the new Armory. This could probably be pulled via the PHP script, but since the file is several thousand lines long, I found it more efficient to save the file first and read it locally.
Please pay special attention to the paths, they will need to be altered in order to work correctly.
#!/bin/bash #Replace YOUR_GUILD_NAME_HERE with your guild name. If your Guild Name is two or more words, it should be in the format #of Your%20Guild%20Name wget --directory-prefix=/path/to/your/desired/directory/ http://us.battle.net/wow/en/guild/YOUR_SERVER_HERE/YOUR_GUILD_NAME_HERE/roster mv /path/to/your/desired/directory/roster /path/to/your/desired/directory/roster.html php /path/to/php/file/ParseRoster.php
The SQL Dump:
Import this into a MySQL database.
-- -- Table structure for table `roster` -- CREATE TABLE IF NOT EXISTS `roster` ( `id` INT(11) NOT NULL AUTO_INCREMENT, `name` VARCHAR(255) NOT NULL, `race` VARCHAR(255) NOT NULL, `class` VARCHAR(255) NOT NULL, `level` VARCHAR(255) NOT NULL, `rank` VARCHAR(255) NOT NULL, `ap` VARCHAR(255) NOT NULL, `prof1name` VARCHAR(255) DEFAULT NULL, `prof1value` VARCHAR(255) DEFAULT NULL, `prof2name` VARCHAR(255) DEFAULT NULL, `prof2value` VARCHAR(255) DEFAULT NULL, `talent1` VARCHAR(255) DEFAULT NULL, `talent2` VARCHAR(255) DEFAULT NULL, PRIMARY KEY (`id`) ) ENGINE=MyISAM DEFAULT CHARSET=latin1 AUTO_INCREMENT=1 ;
The PHP Backend:
The PHP file should be fairly straight forward.
A few notes:
- I have a config file that holds information for my database, if you have a similar file you should include it, otherwise add in the proper mysql_connect() information.
- Make sure the path to the Roster.html file is correct.
< ?php /** * This script will parse the new WoW Armory without an XML file. * This script will currently pull the Name, Level, Class, Race, * Achievement Points, Professions, and Talents of every member * In a specified guild. The script works for me but may not work * as expected on every system. Use at your own risk. * * @author Josh Grochowski (josh[dot]kastang[at]gmail[dot]com) * */ set_time_limit(8000); include("/path/to/config/file.php"); getRosterInformation(); function getRosterInformation() { $roster = file_get_contents("/path/to/roster/file/roster.html"); $dom = new domDocument; $dom->loadHTML($roster); $dom->preserveWhiteSpace = false; //The first tbody tag marks the start of the actual //'roster' part of the html. $roster = $dom->getElementsByTagName('tbody'); //Each Character has its own tr block. $char = $roster->item(0)->getElementsByTagName('tr'); foreach ($char as $c) { //Character information is split into individual //td blocks. $charInfo = $c->getElementsByTagName('td'); $charImages = $c->getElementsByTagName('img'); //I only care about active characters. Inactive characters //will display 0 Achievement points. if((int)$charInfo->item(5)->nodeValue > 0) { $name = $charInfo->item(0)->nodeValue; $race = $charImages->item(0)->getAttribute('src'); $class = $charImages->item(1)->getAttribute('src'); $level = $charInfo->item(3)->nodeValue; $rank = trim($charInfo->item(4)->nodeValue); $ap = trim($charInfo->item(5)->nodeValue); //Returns an array containing the professions name/level and //talents of each individual character. $charArray = getCharacterInformation($name); $query = "INSERT INTO roster(name,race,class,level,rank,ap,prof1name,prof1value,prof2name,prof2value,talent1,talent2) VALUES('$name','$race','$class','$level','$rank','$ap','$charArray[profName1]','$charArray[profValue1]', '$charArray[profName2]','$charArray[profValue2]','$charArray[talent1]','$charArray[talent2]')"; mysql_query($query) or die(mysql_error()); //Wait 5 seconds inbetween queries to keep from getting banned from WoW Armory servers. //This can probably be adjusted to three or four seconds, but if you do get banned, it can //last las long as 48 hours. sleep(5); } } } function getCharacterInformation($charName) { //link to characters page on WoW Armory $charInfo = file_get_contents("http://us.battle.net/wow/en/character/eitrigg/".$charName."/simple"); $dom = new domDocument; $dom->loadHTML($charInfo); $dom->preserveWhiteSpace = false; //Profession Names $xpath = new DOMXPath($dom); $profName = $xpath->query('//span[@class="profession-details"]/span[@class="name"]'); //Profession Values $profValue = $xpath->query('//span[@class="profession-details"]/span[@class="value"]'); //Talents $talents = $xpath->query('//span[@class="name-build"]/span[@class="name"]'); $charArray = array("profName1" => $profName->item(0)->nodeValue, "profValue1" => $profValue->item(0)->nodeValue, "profName2" => $profName->item(1)->nodeValue, "profValue2" => $profValue->item(1)->nodeValue, "talent1" => $talents->item(0)->nodeValue, "talent2" => $talents->item(1)->nodeValue); return $charArray; } ?>
Using Python to Parse WoW Server XML.
THIS CODE IS OBSOLETE. Blizzard changed the way information can be grabbed from their servers. Please view my PHP Class which will work.
The below Python script will parse the WoW Server XML file and return status, type, name, and load about a specified WoW Server. It also stores the Server XML file into a cache.txt file so constant requests to the script will not result in an IP ban on Blizzards servers.
Python 2.6+ must be used for this script to function correctly. It will work out of the box with no third party libraries required.
Please change the SERVER_NAME (on line 32) variable to your servers name.
#!/usr/bin/python """ Author: Josh Grochowski (Kastang) This script will parse the WoW Server Status XML file and return information about a specified server. This Python script works for me, but it may not work for you. Use this script at your own risk. Python 2.6+ must be used. """ """ The below four lines assume this script will be loaded onto a webserver. If you are planning to run this script in a CLI enviornment, feel free to remove the below four lines of code. """ import cgitb cgitb.enable() print "Content-Type: text/html;charset=utf-8" print #Imports from xml.etree.ElementTree import parse import urllib, os, time #WoW Server Status Path WOW_XML = "http://www.worldofwarcraft.com/realmstatus/status.xml" #Change SERVER_NAME to the your server. SERVER_NAME = "Eitrigg" #Display information for the Realm information. rStatus = ["Offline", "Online"] rType = ["", "PVE", "PVP", "RP", "RP PVP"] rLoad = ["", "Low", "Medium", "High"] #Modification time of the cache file. modTime = os.stat("cache.txt").st_mtime #Current system time. currTime = time.time() """ If this function is called, the cache file will be updated with the newest information from the WoW Server Status XML file. """ def updateCache(): cache = open("cache.txt", "w") cache.writelines(urllib.urlopen(WOW_XML)) """ Given a Realm name and XML file, it will find the XML string containing information about the specified Realm and return it to the requesting function. """ def findRealm(realm, xml): for l in xml.findall('rs/r'): if l.get('n') == realm: return l return False #Displays information about the specified Server. def printOutput(server): print "Realm Name: " + server.get('n') print "Realm Type: " + rType[int(server.get('t'))] print "Realm Status: " + rStatus[int(server.get('s'))] print "Realm Load: " + rLoad[int(server.get('l'))] """ If the cache file has not been updated for 10 minutes, it will be updated. Otherwise, the existing file will be used to find current information about the specified realm. If the specified Realm is not a valid WoW Realm, nothing will be returned. """ def main(): if (currTime - modTime) > 600: updateCache() xml = parse("cache.txt").getroot() server = findRealm(SERVER_NAME, xml) if server != False: printOutput(server) if __name__ == "__main__": main()
A sample output can be seen below:
# python realmstatus.py
Realm Name: Eitrigg
Realm Type: PVE
Realm Status: Online
Realm Load: HighPush Daily Weather to your iPhone using Prowl
Below is a PHP script that will pull your daily local weather forecast from Weather Bug and push it to your iPhone using Prowl. One requirement is you have the Prowl iPhone app on your iPhone or iTouch. Prowl is $2.99 in the App Store. This script also requires PHP to be installed on your server/desktop machine. I have tested this script with a handful of random Zip Codes around the US. If I happen to come up with any special conditions, I will update the script below.
Before running this script, make sure you change the XXXXX's in $xml to your local zip code and change YOUR_API_KEY_HERE in $prowl to your Prowl API code. You will also need to change includes('API/ProwlPHP') to the location of ProwlPHP on your server.
/** * This script pulls a local XML weather forcast from * Weatherbug. Pushes the current days weather to a * device running prowl. * * This script requires ProwlPHP, and a valid Prowl API code. * * @author Kastang (josh at kastang.com) */ include('API/ProwlPHP.php'); //Change the XXXXX's to your local zip code. $xml = simplexml_load_file("http://feeds.weatherbug.com/rss.aspx?zipcode=XXXXX&feed=fcst"); //Parse the description from Weatherbug. prowl(substr((ltrim(strip_tags($xml->channel->item[0]->description))),0,-22)); //Push the weather to Prowl. function prowl($weather) { $prowl =new Prowl('YOUR_API_KEY_HERE'); $prowl->push(array( 'application'=>'Weather', 'event'=>'Today', 'description'=>$weather, 'priority'=>0, ),true); }
I have this setup daily to run at 8am using cron. Below is an example you should add to crontab if you want to do the same thing:
0 8 * * * /path/to/weather.php