#Old school runescape wiki how to#
#Old school runescape wiki software#
#Old school runescape wiki free#

An API, or Application Programming Interface, is quite complex. For example, parsing a web page and extracting an HTML table of item properties. You write a simple program, give it a list of URLs of web pages and program the functionality to extract the data that you are interested in. Web scraping is a method to extract data from websites - which is usually an automated process. But what is an API? And what is scraping? Some readers may be new to these concepts, so it is important to summarize them. So we know what a Wiki is and how it is powered. However, the new OSRS Wiki is also powered by MediaWiki.

However, the OSRS Wiki admins recently announced that they migrated off Wikia and moved to an independently hosted Wiki. This is a key aspect… and means that the previous Old School RuneScape wiki was hosted on the open source MediaWiki software.

#Old school runescape wiki software#

Wikia utilizes the open-source MediaWiki software to create and manage the wikis they host. Wikia is exceptionally popular, hosting hundreds of thousands of wikis for a huge variety of topics.

#Old school runescape wiki free#

The OSRS Wiki was previously hosted by Wikia (also known as Fandom) which is a free wiki hosting service. It is prudent to first discuss the technology that powers the OSRS Wiki. Remember, a small group of developers host this content free of charge for the community and we should respect their service and data/content.īackground: Wikis, Scraping and MediaWiki API Although there are no specific guidelines on API usage (at the time of writing this post) - exercise good judgment. I asked a question about API and terms of use on the OSRS Wiki which they answered and specified some requirements: 1) Adhere to rate limiting, 2) Use a custom user agent 3) Be reasonable with the number of requests you perform. Adhere to considerate use of their API.Make sure you read and understand this license. The OSRS Wiki Copyright page includes the following information: Except where otherwise specified, the article revisions on these wikis on and after 1 October 2018 are licensed under Creative Commons BY-NC-SA 3.0.

Since we are using their service and data/content, it is important to take two specific things into consideration:

Background: Wikis, Scraping and MediaWiki APIĪs outlined, this post discusses extracting data from the OSRS Wiki.

#Old school runescape wiki how to#

The remainder of this post is divided into sections that discuss how to extract useful data from the OSRS Wiki with a collection of useful examples of API URLs, which can be tested by copy/pasting into a web browser, or added to a program to automate data extraction. Using web scraping: You get less structured, more messy data.

Using the MediaWiki API: You can get more structured, less messy data.This is, in my opinion, much more effective than scraping the website content, as the wiki contents are (somewhat!) structured. However, the technique discussed in this blog post is not really scraping… rather, it leverages the MediaWiki API to query and parse the OSRS Wiki contents. This post discusses how to “scrape” the Old School RuneScape Wiki website (referred to as the OSRS Wiki in the remainder of this post).