Mashups Live on Standards

Abstract

Mashups integrate web services from pre-existing resources, often submitting them to an unforeseen secondary use. This can only succeed if standards, specifications or other types of regulation are maintained. A good understanding of standards helps mashups and their developers along. The chapter illustrates how web standards are organized and takes its readers through a parade of selected standards. Since standardization is a moderately organized environment, standards and their relatives (specifications, APIs, guidelines) are grouped from a users’ point of view, depending on their usages in mashups: web communication, data organization, content types, web queries, and interfaces. As one may assume standards thrive in cultured web land and are poor or lacking in widely uncharted areas. Application examples are included from time to time. The standards review remains open-ended, but it draws the attention of its readers/users to an important and often neglected issue of their work. Some readers may find hints that directly help during mashup creation. All discussed entities come with their URL references (checked on 19/20-04-2012, a few ones on 20/23-09-2012).

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

2.1 Mashups and Their Underpinnings

Mashups live on standards. By definition they integrate existing web resources and draw new services out of them. We expect the resources to comprise things like data, procedures for their combination, and user interfaces that present the mashup output. Clearly, the data sources must conform to their specifications, web services and their APIs as well, web communication is a must, and the user interfaces of mashups have to comply with good practice. Semantic mashups may require specific formats for instance for their metadata and rules. In general mashups have to conform to the local standards and specifications of their application domains. There are many of them.

Let linked data authors explain what this means in practice. They display their specification underpinnings in a drawing that appears in Fig. 2.1 in an abridged version.¹ As a mashup combines several resources, even more standards from the application areas will walk in! In short, one cannot ignore the cohorts of applicable standards, specifications and the like when dealing with mashups.

Fig. 2.1

Specification underpinnings of linked data (shortened)

Naive minds may expect the standards scene to be thoroughly organized, but beware: you may find standards in the wild instead. The cartoon² in Fig. 2.2 makes fun of how standards propagate. As a consequence of opportunistic growth, standard bodies cover their own areas of interest and may even rival each other. This makes life difficult for developers. In other words, in spite of often being blamed as under-regulated, the Internet appears as a highly standardized environment, albeit inhabited by rampant standards.

Fig. 2.2

How standards proliferate

For dealing with web standards and their relatives, keeping up with some confusion factor is obligatory. The confusion resides inside the area. A description can map it, but this will not change the way agents behave in the field. As a continuous empirical structure is missing, readers of this chapter are the more entitled to ask what they care about when being confronted with regulations in the web, in particular with those related to mashups.

Regulation power clearly is a point. Standards and their kinship may regulate behavior to different degrees. Guidelines tend to be soft. They often propose best practices. Offenses may be worse or better than the guideline demands. In other cases a standard or protocol transgression will malfunction.

Less important is how a regulation is named. Depending on their emitters and their purpose, standards are often labeled specifications, protocols or recommendations. Internet Engineering Task Force (IETF)³ standards are called RFCs (“Requests for Comment”). Famous examples are RFC 791, the Internet Protocol (IP) and RFC 1738, the URL.⁴ The long RFC lists also illustrate the high number of Internet standards. Nota bene that IETF is not the only standard body on the net.

Although standards or specifications are assumed to be set top-down by an authoritative body, web reality knows authority-free bottom-up standardization, too. Companies introduce de facto standards with their product or service APIs, or users massively prefer one practice so that crowds set standards. Pragmatic readers with mashups in mind will accept de facto and de jure standards and apply them.

For use in this chapter, the terminology can be simplified: the term standard is used as long as no discrimination from different types of standard-like regulations is required. Less-than-clear facts will inevitably blur terms, so that retracting towards everyday usage like in “be up to standard” is inevitable. WaSP, the Web Standards Project⁵ fights for standards in the W3C-regulated functional core of the web. Why should the degree of organization and clear-cut definition rise with distance from the center?

Probably ISO, the International Standards Organization⁶ is the most prominent provider of standards in a wide range of applications: ISO 26000 deals with social responsibility, ISO/IEC DIS 17826 with CDMI, which is a cloud data management interface, and ISO 11760:2005 specifies a classification system for coals. According to ISO/IEC Guide 2:2004 a standard is

a document, established by consensus and approved by a recognized body, that provides, for common and repeated use, rules, guidelines or characteristics for activities or their results, aimed at the achievement of the optimum degree of order in a given context [19].

Readers will notice the comprehensive definition of ISO standards. It includes all sorts of regulations for common reuse. This book chapter follows it.

ISO standards are developed in working groups of experts. Draft standards go through an approval procedure. If a draft is accepted by two-thirds of its developers and three-fourth of all ISO members, it is recognized as a standard and published. After some time of use, standards often need a revision. HTML may serve as an example. Revisions run from version 1.0 in 1991 to version 5.0 of today.⁷

Other standard emitters follow more or less the same procedure of standard development and agreement by vote/certification. However, most standardizing organizations are less global than ISO, restricting their domain to a country like DIN⁸ in Germany, a discipline like ITU⁹—the International Telecommunication Union—or a specific environment as W3C¹⁰—the World Wide Web Consortium—does.

Standard bodies may cooperate, but this is far from being the rule. A special mention is, however, due to working groups of different standard bodies who cooperate. WebCGM¹¹ exemplifies a recommendation that W3C and OASIS publish together.

Many bodies offer their standards (“open” standards) for free. This is for instance true for ECMA,¹² OSGi,¹³ OASIS¹⁴ and for W3C. Many Apache implementations distribute open standards, too.¹⁵ Open standards may be connected with the open source software movement.

Open (public) standards are preferred.¹⁶ Proprietary standards are less accessible when their content is not known, and if patents or other regulations impose costs. ISO standard documents and those of other providers such as IEEE¹⁷ or US ANSI¹⁸ are sold. Their prices cover standard development costs. That they are paid emphasizes the manifest practical value of standards—agreed specifications—in many technical domains. Standards reduce costs.

Mixed-mode open and licensed standards exist as well. MP3¹⁹ is a common example.

Standards can support mashups straightforwardly, but just as frequent is a mediated use via an API (Application Programming Interface) that applies standards.

On the value of web standards, one can rephrase the argumentation of the WaSP project cited above: Where specifications and standards reign the web, communication runs better. Since communication is a main feature of the web, web users and developers more and more appreciate the adherence to standards, for well-known common-sense reasons:

• If you follow a best-practice standard, this eases your own development work. You avoid reinventing the wheel.

• Access to your work is eased. Who knows the standard has better chances to reuse your work. This is the well-known accessibility aim for web data and services.

• Software and data conforming to standards can be shared over different hardware and software platforms. This feature is often called interoperability.

Mashup developers feel the advantage of standard compliance even more than other netizens, because their fate is combining resources that may come from far away in the web and that never were anticipated to meet in a mashup. These secondary uses are hard to achieve unless the integrated resources keep to their specifications. As essential underpinnings of mashups, web standards merit a closer look.

2.2 Grouping Mashup-Relevant Standards

Organizing standards is a problem because one cannot refer to an inherent order of the domain. Even big players may present their own standards output in a partially adequate order. W3C is a good example. They are sorting their standards by technology, status, date, title, working/interest group, or editor, and alphabetically in second sort position.²⁰ This is helpful, but far away from a systematic order. When mashups and mashup-relevant standards are of interest, one would also prefer an order that adapts to the newly set focus, in this case to mashups.

Mashups are widely distributed on the web. Especially innovative evolving web ecosystems tend to be less penetrated by standardization. For instance, look at the intermediate regularization state in the clouds.²¹ The author uses cloud services in iPad apps without a need for specific cloud standards. When the authors of [12]²² describe their Web of Things experience with cloud services, they focus on the EPC (Electronic Product Code) network problems.

The W3C incubator group²³ sees social media in a pre-standardization data-silos state, because the bulk of specifications comes from the proprietary APIs of Facebook, Twitter, Google, LinkedIN and others, with their separated networking sites. Nevertheless the group lists many approaches with standards background, e.g. XMPP and OAuth mentioned below, so that social media standardization appears to be in progress. Both XMPP and OAuth are shared with other application areas. Social media mashups combining two independent social networks are coming up.

In the same vein the AR (augmented reality) community is currently setting up standards for their field, repurposing many existing standards and following a systematic development procedure.

At least until the main standardization bodies offer authoritative arrangements, web users are summoned to order standards from their own viewpoint. Many options are possible; there is all reason for pragmatism and none to find out “was die Welt im Innersten zusammenhält”. Grouping mashup-relevant standards for mashup developers and users must reflect the use situation: What do we use a standard for? In which context, with which regulation goal does it appear? In short the question is: Which standard when for doing what? Thus the ordering follows function:

• 1. Web communication. Almost all mashups draw resources from the web or write data to some external server. A web transport is needed, the standards/protocols for web transport apply.

• 2. Data organization. Data on the web must be organized according to preset schemes and syntaxes. This is a precondition for interoperability, i.e. assuring that resources can be interpreted elsewhere in the web as intended by the data producer or the producing system. The regulations may affect small units like characters, but also large ones like whole documents or texts. In user eyes they may appear as formatting schemes although developers may see them e.g. as protocols.

• 3. Content types. As demonstrated by XML formats or Java programs, the web is widely text-based. Life conditions change when one dares into content. Dealing therein with text, images, videos, audio and so on is a key task of semantic mashups. The relevant standards are considered starting with the highly common RDF format and ending with much more complex cases such as augmented reality standardization.

• 4. Web queries. Search is a frequent mashup activity. Especially linked data mashups connect via SPARQL searches. Query standards and specifications are needed.

• 5. Human interfaces. Designing helpful human interfaces is a core effort in mashup development and user-own mashup configuration. In contrast with other tasks, interface design is mainly regulated by guidelines. Companies (Google, Apple, etc.) provide them and put big investments in. Guidelines prescribe features and often support developers with ready-made templates. Since interfaces are near to users, maintaining a local look-and-feel is of evident interest.

The upcoming web standards parade requests some flexibility for avoiding common dead ends. Two of them:

• The one type only trap. While German Wikipedia²⁴ describes MP3 as a procedure of data compression (“Verfahren zur verlustbehafteten Kompression digital gespeicherter Audiodaten”), English Wikipedia²⁵ sees it as an encoding format (“a patented encoding format for digital audio which uses a form of lossy data compression”). As MP3 is a German invention (the author met the main researcher Hans-Georg Musmann), the German version may be more authentic. Anyhow users are invited to switch types to the roles required in their task environment, without further discussion.

• The fixed surroundings trap. In their standards organization, W3C present the XML Document Model DOM²⁶ as a subcategory of ‘Scripting and AJAX’. Why? To the author it is known from its Java implementation in JDOM.²⁷ Thus transferring items into new environments is approved—everybody can quote precedents.

There is nothing wrong when for instance a developer-side protocol like SOAP appears on the user side as a format for the protocol execution, according to the user’s tasks. Similarly cooperative users of standards look over the fence and reuse what was invented elsewhere on the web. Context-driven switches of type, affiliation and other properties are normal.

2.3 The Web Standards Parade

In the following main section of the chapter, mashup-relevant standards parade in groups as listed above. Because this chapter belongs to a book on semantic mashups, content-related standards get their due share of attention. Depending on the local situation, the discussion centers on specific areas (e.g. security) or on concrete standards (such as XML).

2.3.1 Web Communication

2.3.1.1 HTTP and TCP/IP

HTTP and TCP/IP manage web communication in a client–server environment. The Internet Protocol Suite TCP/IP [9] includes the Transmission Control Protocol (TCP) and the Internet Protocol (IP). It includes several layers (see Fig. 2.3 adapted from [18]) reaching from network transmission to application services such as FTP or SMTP. The basic truth is that every computer in the web owns an IP address. IPs are provided by the Domain Name System (DNS).²⁸

Fig. 2.3

The TCP/IP architecture

HTTP²⁹ (Hypertext Transfer Protocol) is currently used in its HTTP 1.1 version. It opens reusable connections between two hosts. As well known, HTTP addresses show up in the headline of the browser when it is directed to a website. Everybody knows YouTube:

http://www.youtube.com/

An HTTP header may also contain some additional items. In the example

http://www.f5.com:80/path/time.jsp?h=13

the port 80 is specified, the data type is jsp (Java Server Pages), and after the question mark follows a short query. Web addresses are commonly called URLs (Unit Resource Locations). They verbalize numeric IP addresses, for instance

http://17.149.160.10/

is transformed into

http://www.apple.com/

The most common HTTP requests are GET and POST. In general, users will apply them from inside an HTML form.

2.3.1.2 REST (Representational State Transfer)

The most popular style to address a web host is REST (Representational State Transfer) ([11],³⁰ [31]). Look at a REST example of my own use, a request string fetching web speech data that go straight to the audioplayer:

REST³¹ is built on web resources that are identified through URIs as usual on the web. It connects with HTTP as a service interface. In a typical REST request, the client discovers the URL of a service, it sends an HTTP call to this URL with a given HTTP command (GET, POST, PUT, etc.), a number of options (e.g., accepted format), and a payload in the negotiated format (e.g., in XML or JSON). In my example an audio file is returned.

Developers (the author included) like REST because its is easy to use. No official standard is needed to impose it as a standard approach [13].³²

2.3.1.3 SOAP (Simple Object Access Protocol)

SOAP is defined in a W3C standard.³³ According to a SOAP tutorial,³⁴

SOAP is a simple XML-based protocol to let applications exchange information over HTTP.

Sometimes this stance rises smiles in the web,³⁵ but SOAP has its defenders and users.³⁶ ProgrammableWeb of today (08-02-2012) records 904 Mashup APIs using SOAP while 3433 APIs use REST. A simple SOAP example points out why SOAP is less simple than REST use. For getting a Fahrenheit conversion of a Celsius degree one has to code and send a SOAP envelope:

The answer arrives in an XML formatted envelope:

After unpacking the XML, the value of 122 degrees Fahrenheit is available.

2.3.1.4 Syndication: RSS (Really Simple Syndication) and Atom

All sorts of syndicated news, audio and video podcasts are streaming through the web, often in two popular feed formats in parallel: RSS 2.0.³⁷ and Atom.³⁸ RSS 2.0 is more frequently used, but the newer Atom format provides a larger hierarchy and more tags.³⁹ Under their XML head both formats organize a series of items in a hierarchy. The syntax prescribes the tag names and their sequence. Tools like parsers and validators are available.

RSS 2.0 defines information channels for its feeds. Channels have some header information followed by a sequence of items. For the sake of simplicity only one of them is shown in the following example:⁴⁰

The following sample Atom feed from the same source exemplifies the more detailed Atom feed format:

2.3.1.5 Extensible Messaging and Presence Protocol (XMPP)

The Extensible Messaging and Presence Protocol (XMPP) [32]⁴¹ is an open technology for real-time communication, using XML as the base format for exchanging information. In essence, XMPP provides a way to send small pieces of XML from one entity to another in close to real time.⁴²

XMPP builds on a series of IETF RFCs⁴³ and a list of extensions⁴⁴ to them.

XMPP core services provide channel encryption, authentication, presence check for web entities, messaging, service discovery, structured data forms, and workflow management.

XMPP is widely deployed for instant messaging, chat, data syndication and other application types. Main sponsors are Google, the US Secure Hosting Center, and Voxeo.

2.3.1.6 Security: Encryption and Authentication

Internet security has evolved to an own field of investigation [20].⁴⁵ It may be a very serious concern in web communication. Among other things, hash encryption methods have been developed for secure Internet interaction. In critical tasks such as authentication they may even be obligatory.

The common encryption standards family is SHA (secure hash algorithm).⁴⁶ The SHAs are US Federal Information Processing Standards (FIPS).⁴⁷ SHA-1 and SHA-2 are in use, SHA-3 is currently being developed. MD5⁴⁸ was standardized by the Internet Engineering Task Force IETF in 1992.⁴⁹ It is frequently applied, but it has been found to be unsafe.

Authentication means identifying a computer user, authorization decides what the user is permitted to do. For users it is an evident advantage being easily admitted to resources of others on different sites, in particular in social networks. Just as evidently the security concerns of the resource owner must be protected.

Using OpenID⁵⁰ or OAuth,⁵¹ users can authorize others to access their private resources without handing over their credentials to a foreign site. As often an image says more than a thousand words. Figure 2.4 ⁵² explains how the two competing authentication/authorization services OpenID and OAuth ensure that only entitled users can access a resource. To admitted users, OpenID hands over a certificate, whereas OAuth provides a time-restricted key. Practical how-to advice for developers is offered e.g. by Google.⁵³

Fig. 2.4

OpenID and OAuth authorize a user

2.3.2 Organizing Data

Depending on web sectors different data formats are prevailing. JSON is a popular choice. The CSV (Comma-Separated Values) format is used frequently. Everybody agrees on them, but the main formats of the semantic web are different: HTML, RDF, and XML are the basic big three. They are all specified by a specific markup language. In contrast with HTML and XML, RDF is conceptual, with no implementation of its own.

Data standards of interest for the semantic web and semantic mashups are discussed advancing from simpler to more complicated formats. HTML comes first, whereas the recent and much more comprehensive HTML5 is in the end group.

2.3.2.1 HTML

The most visible items on the web are the websites. Basically they are coded in HTML, the Hypertext Markup Language. HTML is specified and standardized by the W3C consortium.⁵⁴ Till HTML 4, style sheets such as CSS (Cascading Style Sheet)⁵⁵ or JavaScript⁵⁶ help to adapt HTML web content to the target presentation. For normal use, browsers (there are many of them: Chrome, Firefox, Internet Explorer, Opera, Safari, etc.) display HTML-style web content fetched from web servers. Web pages may be static or dynamic, i.e. being generated at question time from a database with the help of some code (often in PHP or AJAX). Browsers, document editors and so on may host plugins for handling special data types, e.g. video or PDF. HTML tutorials are legion on the web.

2.3.2.2 XML

The XML family⁵⁷ is at least as important on the web as HTML and company. XML is a main carrier format for structuring/markup of web data. Today’s XHTML⁵⁸ illustrates a common XML specialization. It is XML-based, i.e. it complies to the XML syntax, but it specializes it for its own HTML-style purposes. There are many application-specific XML markup languages behaving like this. For instance XML codes SVG graphics as well as the widely used syndication formats RSS 2.0 and Atom.

The XML example below reminds you of the basic syntax of XML: an XML declaration in the header line followed by a hierarchy of tagged entries with tags of your choice.

In XSL,⁵⁹ the extensible stylesheet branch of the XML family, the most well known member is XSLT⁶⁰ for modifying XML structures.

XML Schema (XSD)⁶¹ defines the structure of XML documents. Who wants to get along with less effort tries Relax NG⁶² or the older DTD (Document Type Definition).

A human-friendly alternative to RDF/XML is offered by Turtle (Terse RDF Triple Language)⁶³ now in W3C draft status.

In the semantic web, much data representation builds upon the XML grounding. Indeed XML is—together with RDF—a backbone of the semantic web.

2.3.2.3 RDF (Resource Description Format)

RDF⁶⁴ models concepts in a subject–relation–object pattern. The assumption is that all information can be broken down into triples of this format, corresponding to simple sentences. According to the assumption, all objects can be characterized with combined RDF-based descriptions. The basic RDF format is accompanied by RDFS schema for setting up simple concept relationships, and by RDFa for putting and finding RDF triples in running HTML text.

A classical drawing (Fig. 2.5) from the RDF Primer⁶⁵ explains what RDF contributes. Eric Miller is first characterized by a graph, then an XML-based description.⁶⁶ In the drawing, yellow squares represent strings, green ovals URLs/URIs. Remark that links are defined by URIs, too. In the XML format this information is mapped to tags and content items. A concept is characterized by as many RDF descriptions as needed.

Fig. 2.5

Remake of the classical RDF demo graph: Eric Miller

2.3.2.3.1 RDFS—RDF Schema

A set of RDF triples or concepts is a vocabulary. Therein some organization is required in order to locate basic RDF triples or concepts built from them. The relations for that are provided by RDF Schema (RDFS),⁶⁷ the RDF Vocabulary Description Language. Again a standard example⁶⁸ (abridged) shows what happens. Descriptions of classes are located in the hierarchy with relations such as ‘subClassOf’. In the example the passenger vehicle is specified as a subconcept of the vehicle because a ‘subClassOf’ link attaches it to its hypernym.

2.3.2.3.2 RDFa

RDFa⁶⁹ is about RDF attributes in HTML or XML documents. So far RDF has been presented as mounted on XML. This is common, but not compulsory. The following RDFa⁷⁰ code fragment is thought to sit in an XHTML/HTML5⁷¹ file. It enters RDFa-own tags like properties or attributes into XHTML items, here into a ‘div’ element. With its prefix ‘dc’ the property attribute refers to DublinCore.⁷²

RDFa attributes can be recovered from XML/HTML code. One major approach is gleaning them.⁷³ Practical applications are Google rich data using RDFa and the related microformats.⁷⁴

2.3.2.4 DOM (Document Object Model)

W3C DOM is a standard for accessing HTML and XML documents:⁷⁵

The W3C Document Object Model (DOM) is a platform and language-neutral interface that allows programs and scripts to dynamically access and update the content, structure, and style of a document.

The DOM recommendations are separated into three different parts/levels: core, HTML and XML.⁷⁶ Best known is the XML DOM API. It accesses the hierarchical XML document tree with all its nodes for parsing, writing and restructuring. As many XML documents run through the web, there is a considerable demand for XML DOM parsers. Off-the-shelf tools in several programming languages are available, and also helps for own parser development.

2.3.2.5 JSON (Java Script Object Notation)

JSON is a very common data format. Its language-independence and simplicity enhances its popularity. JSON is a subset of the JavaScript Object notation. JavaScript is supported by virtually all browsers. ECMA standardizes JavaScript under the official name ECMAScript.⁷⁷

JSON works with name/value pairs. Values may be numbers, strings, booleans, arrays, objects, and nil. In the introductory W3C example three employee objects are included in a larger object named “employees”:

JSON objects can be accessed in JavaScript and in other programming/scripting languages.

2.3.2.6 Linked Data [15]

Linked Data⁷⁸ are RDF triples from anywhere with a link in between. Because the triples and their links are identified by URIs, the linked data structure is a well-defined graph in the web. The triples represent concepts or things (not only documents), so that the web switches to a web of data or a web of things (WoT). Figure 2.6 ⁷⁹ shows a small subset of the global linked data space [14]⁸⁰ with the main player DBpedia [4]⁸¹ and some resources it refers to. With their impressive size in the web linked data set standards by fact. Linked data are recognized by W3C without explicit standard/recommendation.

Fig. 2.6

Linked Open Data—small sample of the data network

According to the advice of Berners-Lee⁸² you support the linked data movement by conforming to four simple rules:

• 1. Use URIs as names for things.

• 2. Use HTTP URIs so that people can look up those names.

• 3. When someone looks up a URI, provide useful information, using the standards (RDF*, SPARQL).

• 4. Include links to other URIs, so that they can discover more things.

For more you may want to watch a video lecture⁸³ of Tom Heath and colleagues.

2.3.2.7 OWL (Web Ontology Language)—A Biomedicine-Focused View

Ontologies are structured vocabularies. They reflect the semantics of their application domain. Basic links like the generic ‘is-a’ relation are borrowed from traditional thesauruses and library classification schemes. These also received an RDF-based web implementation and specification: the Simple Knowledge Organization System SKOS.⁸⁴ Metadata often refer to DublinCore specifications.⁸⁵

Readers who doubt the impact of ontologies are invited to the National Center for Biomedical Ontology (NCBO).⁸⁶ Their BioPortal⁸⁷ gives access to a wealth of ontologies in biomedicine, from the Adult Mouse Brain (ABA) to the Zebrafish Anatomy and Development (ZFA). These and other ontologies set annotation standards for their scientific areas.

In biomedicine we find an environment of ontological tolerance. Methodological underpinnings differ. BioPortal supports ontologies in OBO format, OWL, RDF, Rich Release Format (RRF), Protégé frames, and LexGrid XML.⁸⁸ Look at a sample concept in the OBO format of Gene Ontology (GO):⁸⁹

OWL, the Web Ontology Language⁹⁰ is the most-cited ontology markup language of the semantic web. By matter of fact, Protégé⁹¹ is the near-to standard editor for OWL ontologies. A large set of OWL ontologies is available from the Protégé Ontology Library.⁹²

OWL 2 contains three degrees of specification: OWL DL and OWL Lite semantics are based on Description Logic, while OWL Full uses a more extended semantic model. Figure 2.7 illustrates the layered OWL setup. Currently, OWL Full waits for larger future applications.

Fig. 2.7

OWL integration layers

OWL follows a concept import strategy. Constructs from RDF, RDFS and Description Logics (DL) [1] are reused. They are applied as-is or—in the case of DL—adapted to their new environment. Some subsidiary OWL constructs cover concepts for which no suitable import items are available.

OWL integrates the sophisticated concept building of Description Logics so that concepts can be constructed from their semantic features. The very popular example for demonstrating the effect is the grandmother (Fig. 2.8). Note that she is set up from highly recyclable concepts/features only: ‘female’, ‘male’, ‘person’, ‘parent’ and the relation ‘has.Child’. It is easy to imagine how such a reduction to basic concepts can simplify e.g. medical vocabularies.

Fig. 2.8

Description Logic: Grandmother constructed from semantic features

The following code sample taken from the OWL Primer⁹³ illustrates how constructs from RDF and RDFS work together with additional OWL tags: the OWL class ‘Father’ is built with imported ‘rdf:about’ and ‘rdfs:subClassOf’ links and an OWL-own ‘intersectionOf’ with roots in DL. Comparing the DL version of the ‘Father concept’ in Fig. 2.8 above with its notation below roughly illustrates how DL notation is transferred into an XML/RDF/OWL version.

As mentioned before, the standard editor for OWL ontologies is Protégé. On the Protégé screen (Fig. 2.9), the minimal family first appears as hierarchy with additional relations that came over from DL. The second view shows a graphical visualization. The editor builds the RDF/XML code of OWL.

Fig. 2.9

Protégé views of OWL ontology: Hierarchy on the left, visualization as graph on the right

2.3.3 Content Types

The web is said to be text-based, as opposed to sound-based or image-based, although much web content uses media like image or audio. The text-based core of the web consists of organized character sequences (often encoded in ASCII) carrying core web information as in HTML pages, Java programs, Python scripts, etc. They serve technical purposes—their readers are in first line compilers and interpreters, humans reading them are mainly programmers.

Since earlier times, the contrasting non-ASCII MIME (Multipurpose Internet Mail Extensions) type covers attachments like pictures that emails transport as attachments. Iana⁹⁴ registers MIME media types like audio, text, image and 3D models.⁹⁵ The way how MIME types are declared is standardized as well, e.g. as “image/png” or “audio/mp4”.

Graphics MIME types refer both to 2D and 3D dimensioning (overview in [22]⁹⁶). In 2D, raster graphics or bitmaps are composed of arrays of pixels while vector images describe the paths that will show up in the image. 3D graphic uses 3D models—3D representations of geometric data. Web 3D would refer to all technologies that enable 3D applications in the web. For them, the Web3D Consortium⁹⁷ is the first-line reference.

A contrasting view on how to manage web text as a content type of its own is held by the Text Encoding Initiative (TEI).⁹⁸ The Digital Library of Georgia⁹⁹ displays an extensive list of content-oriented media types. The Dublin Core DCMI Type Vocabulary¹⁰⁰ may also offer a suitable term.

In mashups, media types are used for transmitting content. In case of doubt, imagine mashups without the popular maps. Media transport content—this is what semantic mashups interpret. Without any claim for completeness, the following content media are considered:

• 1. PDF (text + image + more)

• 2. PNG (image)

• 3. SVG (image)

• 4. OpenGL (2D, 3D graphics)

• 5. X3D (interactive 3D)

• 6. MPEG’s MP3 (audio) and MP4 (video)

• 7. Maps

• 8. HTML5

• 9. AR (augmented reality)

2.3.3.1 PDF (Portable Document Format)

The great merit of PDF is the platform-independent representation of text and image documents. PDF was proposed by Adobe and still keeps features of a proprietary format. The PDF ISO standard 32000-1 released in 2008 refers to Adobe PDF 1.7.¹⁰¹ Altogether the ISO Store¹⁰² offers 13 separate PDF Standards.

PDF is widely used and has considerable industrial support.¹⁰³ Special purpose versions of PDF have their own standards background, as the PDF/Xs¹⁰⁴ that targets different tasks in the printing industry.

2.3.3.2 PNG (Portable Network Graphics)

PNG (Portable Network Graphics) is a bitmapped image format.¹⁰⁵ It improves over its predecessors GIF and TIFF.¹⁰⁶ Images in PNG format are smaller than in TIFF and therefore more appropriate for web use. With respect to GIF, PNG performs better on alpha channels (variable transparency), gamma correction (cross-platform control of image brightness), and two-dimensional interlacing (a method of progressive display). PNG also compresses better than GIF, and in contrast to GIF it is available without fees because no patents are used. PNGs are processed almost everywhere where bitmaps are expected. They compress well without losses. However, PNG does not support non-RGB color spaces such as CMYK so that PNGs must be converted for printing.

2.3.3.3 SVG (Scalable Vector Graphics)

Scalable vector graphics¹⁰⁷ are two-dimensional. All modern web browsers render them to different degrees. Graphics behavior is stated in XML files so that developers can specify and update them on a text editor.¹⁰⁸ The following code sample illustrates the SVG XML representation for the beginning of the TCP/IP drawing in Fig. 2.3.

2.3.3.4 OpenGL (OpenGraphics Library)

OpenGL¹⁰⁹ is a royalty-free platform-independent industry standard serving both 2D and 3D graphics development. OpenGL is widely used in CAD, virtual reality applications, visualization and computer games, also on mobile devices. In the author’s working context it appears on iPhone and iPad. As in earlier cases we observe on the Khronos site¹¹⁰ a whole family of standards, this time of industry specifications for the web, serving also embedded systems as WebGL¹¹¹ does.

2.3.3.5 X3D (Extensible 3D)

X3D¹¹² is an ISO markup language standard for defining and communicating real-time, interactive 3D content for visual effects and behavioral modeling. It is replacing the Virtual Reality Modeling Language (VRML). X3D uses XML and integrates with relevant other technologies (e.g. MPEG-4 or SVG). According to its authors X3D is royalty-free, scalable and open. The fit into HTML5 is explored by x3dom.¹¹³

X3D is on its way to current browsers. Serving the tiny piece of code in Fig. 2.10 to Chrome equipped with instantreality¹¹⁴ shows you two blue triangles (on the right of the figure) that you can move around.

Fig. 2.10

X3D code (left) and its result on the Chrome window (right)

2.3.3.6 MPEG—Motion Pictures and Audio Standardization

The Moving Pictures Expert Group (MPEG)¹¹⁵ is a major player on audio and multimedia specifications. According to their own mission statement,¹¹⁶ their area of work is the

Development of international standards for compression, decompression, processing, and coded representation of moving pictures, audio, and their combination, in order to satisfy a wide variety of applications.

Their list of standards¹¹⁷—delivered via ISO or under way—is just as impressive as those of other big providers.

The main influences on web audio and video practice come from the standards MPEG-1, MPEG-2 and MPEG-4. They are the technical background of MP3 audio and MP4 digital video. Television and radio rely on them. Internally the mentioned standards are bundles of specifications. Their inside view is displayed on the MPEG site. Figure 2.11 ¹¹⁸ provides a minimal orientation of the relationships between the audio and video standards of our interest. VCD is video CD, DVD is optical disk, DVB is digital video broadcasting. DVB-S2 is its satellite-based follower of the second generation certified by the European Telecommunications Standard Institute ETSI¹¹⁹ in 2005. PS is the program stream defined in MPEG-1, TS is the transport stream. An overall introduction to video, DVD, and digital sound is available at karbosguide.¹²⁰

Fig. 2.11

Main relationships of audio and video standards

MPEG standards are charged with patents and engender license fees.¹²¹

MPEGLA¹²² manages them, but not all. On their patent pools site¹²³ they tell that they stopped dealing with MPEG-4, but anyway they list many companies that are in with specific patents. MPEG-4 Visual is still managed by MPEGLA with “reasonable” (sic) royalties.

The situation of MP3 patents is just as unclear. MPEGLA asserts to manage them. However, from mp3-tech¹²⁴ one learns that only Fraunhofer/Thomson patents require royalties, with a link to their price list. It is easy to imagine that the many involved companies litigated on their deals and rights. Indeed US courts decided on huge indemnities.

Although MP3 is widely used, it has to share the market with some alternative options. A few of them:

• WMA—Windows Media Audio¹²⁵

• WAV—Waveform Audio¹²⁶

• AIFF—Audio Interchange File Format¹²⁷

• MIDI—Musical Instrument Digital Interface¹²⁸

• Ogg Vorbis—a free, open and un-patented music format¹²⁹

Remarkable is the Chinese Audio Video Standard AVS.¹³⁰ According to its owners, it is faster than MPEG-2 and MPEG-4. It is owned by China and frees Chinese developers from MPEG royalties.

2.3.3.6.1 MP3

MP3 is a de facto standard for consumer digital audio. It is an audio compression technology based on the MPEG-1 (lossy audio and video compression) and MPEG-2 (update and extension) specifications, referring to the MPEG Audio Layer 3. MP3 compresses CD quality sound by a factor of 8–12, while maintaining almost the same high-fidelity sound quality. MP3 compression loses sound depending on the compression rate, but it loses intelligently:

• It keeps the audible audio data at the cost of less audible ones.

• Loud sounds drown softer ones, so that the softer sounds are removed.

• Human ears hear best in the range of 1–4 kHz. Sounds in this zone remain unchanged, while higher or lower ones may by reduced.

• In the marginal areas, common information of stereo sounds may be reduced to mono sound.

Systems apply many additional compression strategies, and often combine them. [26]¹³¹ reviews the digital audio standards. Who wants to learn more about the effort that lead to MP3 can read the account of a highly involved Hanover scientist [25].

An MP3 file is made up of multiple MP3 frames, which consist of a header and a data block. The MP3 data blocks contain the (compressed) audio information in terms of frequencies and amplitudes. Today most MP3 files contain ID3 metadata¹³² that precede or follow the MP3 frames.¹³³ Frames are fed into each other in a Matryoshka style [30].¹³⁴

2.3.3.6.2 MP4

As an MP3 file extension makes think about music or audio, MP4 makes think about video. Again a drawing (Fig. 2.12 ¹³⁵) conveys an initial orientation, this time of what works inside an MP4 file. Additionally it explains that MPEG-4 grew out of Quicktime. Although users perceive MP4 as an integrated multimedia technology, a whole standards family stands behind it. MP4 or MPEG-4 AVC (Advanced Video Coding)¹³⁶ is a container format. It can contain all kinds of multimedia content—like audio, video, 2D and 3D graphics and animated avatars, besides user interactivity features. MPEG-4 stores audio, scenes (described by the scene language BIFS), and other multimedia content using the ISO Base Media File Format, while AVC means the storage of AVC (ISO/IEC 14496-10/AVC) standard 4 data (which may be audio, video, or sub-scenes) within the files of the ISO Base Media File Format.¹³⁷ MP4 containers are box-structured files.¹³⁸ A box-structured file consists of a series of boxes that have a size and a type. The file structure is object-oriented. All box-structured files start with a file-type box (possibly after a box-structured signature) that defines the best use of the file, and the specifications to which the file complies.

Fig. 2.12

The standards background of MP4

The files have a logical structure, a time structure, and a physical structure:

• The logical structure of the file is that of a movie which in turn contains a set of time-parallel tracks.

• The time structure of the file is made of the temporally ordered samples in the tracks.

• The physical structure separates the data needed for logical, time, and structural de-composition from the media data samples themselves. The structural information is concentrated in a movie box.

Each media stream is contained in a track specialized for that media type (audio, video, etc.), and is further parameterized by a sample entry. The sample entry contains the name of the exact media type (i.e., the type of the appropriate decoder) and any parameterization as needed by decoders.¹³⁹

2.3.3.7 Maps

The most popular media to be mashed into an application are maps, possibly superimposed with location information on persons and all sorts of things, e.g. businesses. A recent map site reminds you of the information value of map media (Fig. 2.13). It displays tonight’s program of Chicago theaters,¹⁴⁰ nicely exemplifying the attractivity and the information value of map-driven user services. On the hit list of digital map delivery, Google Maps¹⁴¹ is most probably number one, followed by Yahoo maps¹⁴² and other providers.¹⁴³ ProgrammableWeb knows of 2351 mashups using Google Maps.¹⁴⁴ Additional evidence for the popularity of digital maps comes from related listings¹⁴⁵ on the web.

Fig. 2.13

Theaters in Chicago—users see what is on stage in the district

Who wants to integrate a Google map into the own application, applies the Google Maps API for web services.¹⁴⁶ First results are easily achieved, so that complying with the Google Maps API specification is immediately rewarding. Please register that map standardization is driven by industrial deployment.

Correspondingly to the thriving application area the realm of map services standards is flourishing. The Open Geospatial Consortium (OGC) presents an impressive list,¹⁴⁷ containing the Web Map Service¹⁴⁸ and many other standards. The Web Map Service Interface Standard (WMS) therein provides a simple HTTP interface for requesting geo-registered map images from one or more distributed geospatial databases. It returns geo-registered map images that can be displayed in a browser.

Keyholes are points on earth defined by longitude, latitude and altitude for the use in Google Earth and other applications. The respective Keyhole Markup Language (KML) is developed by Google and standardized by OGC.¹⁴⁹

Points of interest (POI) are locations that may be important for somebody for some reason. Usually they are defined by their geographic longitude and latitude. They obtain scientific interest as well, but interestingly a standard for points of interest is still missing. GPS provides an industrial specification.¹⁵⁰ Commercial and free collections of POIs are available.

The Geography Markup Language (GML)¹⁵¹ standardizes an XML-based representation of geographical features. A code sample from the GML 3.1 specification¹⁵² displays how an XML/XSD configuration works for geographic observations:

2.3.3.8 HTML5

After good old HTML, HTML5¹⁵³ is a big jump forward. HTML5 is still a W3C candidate recommendation,¹⁵⁴ but it is already widely applied in the web. There are galleries full of sites coded in HTML5.¹⁵⁵ Some big web companies support HTML5. Take a look at Apple’s HTML5 show page,¹⁵⁶ or at the still more engaged Google HTML5Rocks resource.¹⁵⁷

There we learn the basic HTML5 recipe:

HTML5 = HTML + CSS + JS

Cascading Stylesheets (CSS)¹⁵⁸ for presentation and JavaScript (JS)¹⁵⁹ for client-side scripting are integrated with the HTML functionality, so that with the HTML5 environment alone, users dispose of virtually all means for setting up a site. In addition, HTML5 is equipped with graphics tools—SVG and Canvas. It offers video and audio tags and plays video and audio files without additional software.

A tiny test of the HTML5 advances: Put the following piece of SVG-including code into a file and pass it to your browser. It will draw a green ball.

Apple and Google are not alone in supporting HTML5 developers. HTML5 tutorials abound on the web.

2.3.3.9 Augmented Reality (AR)

Augmented reality (AR) adds some virtual information to a real-world environment. This can be helpful because it makes at a time visible on the same camera screen what otherwise must be looked up separately. Positive effects are evident for instance when a learner sees the real-world problem together with a virtual inset explaining how to solve it.

The field is recent and successful, so that a need for standards is felt. Remarkably standardization is discussed in a collaborative style with integration of existing standards that can be reused for AR: JSON, HTML5, KML, GML, CityGML, X3D can be repurposed. [29]¹⁶⁰ focus on mobile AR. They propose to take over standards from several sources, for example Sensor Web Enablement (SWE) standards maintained by OGC.¹⁶¹ The aim is to avoid mistakes and to speed up the standardization process. Figure 2.14 illustrates how much earlier work can be exploited. At the same time it makes manifest that realizing and standardizing AR is a complex task. Many subresources must be incorporated.

Fig. 2.14

AR standardization with integration of existing standards from [29]

[23]¹⁶² proposes a step-by-step procedure for standards creation:

• 1. Stakeholder analysis

• 2. Architecture analysis

• 3. Scope definition

• 4. Usecase definition

• 5. Breaking it down

• 6. Analyze existing standards and standard development organizations (SDOs)

• 7. Fill the gaps

• 8. Feasibility check

In [24]¹⁶³ the XML-based Augmented Reality Markup Language (ARML) is explained. Its specification is available on the Wikitude developer page.¹⁶⁴

2.3.4 Web Queries

While elsewhere in this chapter web (pre)-history is almost disregarded, this is less defendable for retrieval technologies, simply because the web inherits so much from earlier approaches. After all Edgar F. Codd proposed the relational data model in prehistoric times (around 1970), together with some relational algebra and the retrieval algorithms which were called SQL. In information retrieval the first implemented approaches appear already in the late 1940s.¹⁶⁵ When ANSI and ISO began standardizing SQL for search in relational databases in 1986/1987 they encountered a good state of earlier work. Today SQL development is going on, currently a new working draft is discussed.¹⁶⁶

A timeline of newer web history (from [3]—Fig. 2.15) shows that only after 1995 pre-existing retrieval and query strategies were reworked for use on the web. Web search for structured data adapted the predefined SQL search techniques: SQLite is popular for web databases, XQuery searches XML data, and SPARQL searches RDF graphs. Since RDF data most of the time serializes in XML format, both XQuery and SPARQL apply to RDF data.

Fig. 2.15

W3C standards for XML and RDF—recent timeline. Web query development starts around 1995

2.3.4.1 SQLite SQL

SQLite ([28], [21]¹⁶⁷) is an SQL database engine that embeds into applications—because of its small footprint also on cellphones, PDAs, and MP3 players. It is platform independent, binds to many programming languages and adapts to low memory constraints. SQLite is heavily used as embedded database engine in the web. Wikipedia qualifies it as “arguably the most widely deployed database engine”. Interestingly SQLite achieves this diffusion with a mixed standards behavior—implementing most of the SQL standard, but also specific features of its own.

The author’s moderate experience was acquired on iPhone/IOS5.¹⁶⁸ A select command “as understood by SQLite”¹⁶⁹ (Fig. 2.16) recalls core SQL query syntax and shows how SQLite conforms to it.

Fig. 2.16

SQL select as understood by SQLite

2.3.4.2 XQuery

XQuery¹⁷⁰ is a W3C-specified query language for XML data sources. The whole XQuery family including XPath, XSLT and others is strongly supported by W3C.¹⁷¹ New query expressions and processing models are defined in the XQuery Update Facility 1.0 recommendation of March 2011.¹⁷² Many XQuery tutorials are available.

XQuery is running on web XML databases.¹⁷³ A code example from ipedo¹⁷⁴ (Table 2.1) illustrates the use of XQuery expressions. From a tiny XML bike resource (left), the query statement (on the right) retrieves the red bicycle because its price meets the query condition.

Table 2.1 An XQuery expression selects a bike from XML data

2.3.4.3 SPARQL

SPARQL (Protocol and RDF Query Language) is a W3C recommendation since 2008.¹⁷⁵ It searches RDF-coded data resources, so that one finds RDF triples and their conjunctions, disjunctions and additional patterns. SPARQL has considerable industry support. It is implemented for many programming languages.¹⁷⁶ SPARQL queries can be transferred into SQL and XQuery formats.¹⁷⁷ The opposite direction—exploiting and rewriting non-RDF relational databases in RDF format—is in the implementation state.¹⁷⁸

Protégé includes a SPARQL retrieval technique for OWL ontologies, with the restriction that availability in Protégé 4 is still unclear.¹⁷⁹

DBpedia offers a SPARQL endpoint.¹⁸⁰ The luxury cars example from there (Fig. 2.17) explains all we need for instant purposes. The prefixes show which resources are used. The SQL-style select query uncovers the SQL roots. Who wants to see XML and JSON formats changes the presentation format.

Fig. 2.17

SPARQL luxury cars example—remark prefixes and SQL query expression

SPARQL entailment standards for expanding queries to implicit results are under way.¹⁸¹

2.3.5 Human Interfaces

Human interfaces are the realm of Human Computer Interaction (HCI). As most mashups are made to serve human users, interfaces are a main issue for mashup developers. Long-standing research and HCI classics like [27] and [7] let us assume that developers have a good grasp of interfaces, in particular of Graphical User Interfaces (GUIs). However, today’s user interfaces have advanced over their classical ancestors by being interactive, touch-sensitive, including auditory components, and so on. Media they present to users may vary—from songs to maps, mathematical formulas, soundscapes, webcams or surgical cut videos.

Long-standing HCI knowledge¹⁸² also might entail the assumption that a common understanding of interfaces resulted in generally accepted specifications or standards. According to available evidence this is not the case. Some standards are indeed available¹⁸³ but they are a minority group. [2] presents an overview of the majority inside the minority group: the ISO-based standards.

Interfaces are a guideline country. One reason for the predominance of guidelines is that interfaces communicate with users, so that flexibility, style and esthetics play an important role. Interfaces mediate between users and applications, they must adapt to both sides—the system’s services must be handled so that users succeed in working with them without unnecessary clashes. Design strives for functionality and usability as for recognition value, esthetics and corporate style (the “look and feel”). Users interact with the interface, they have to feel at home with it.

Interface guidelines for general audiences are available, scaling from rule-of-thumb rules to very sophisticated and recommended work.¹⁸⁴ Among newer (e)books one can hint at [33] and [10].

Companies invest massively into web guideline development. This can be observed with IBM: “Good design is good business.”¹⁸⁵ Apple can serve as an example for specializing guidelines to tasks and devices such as Mac, Bluetooth, USB device, iPhone, iPad, etc. Thirty interface-oriented guidelines and references come up from the OS X Developer Library.¹⁸⁶ SAP¹⁸⁷ helps companies to run better, their user-centered design guidelines¹⁸⁸ target business tasks and users. Many other enterprises follow the same strategy for their own businesses or devices.¹⁸⁹

The impact of map services visualization is evident enough to motivate a discussion of geographic information presented according to a few guidelines in different styles [16].¹⁹⁰

Digital map developers are not alone in thinking of specific user interfaces for their media. Audio interfaces ([17],¹⁹¹ [8]¹⁹²)—why not? 3D interfaces ([5], [6]¹⁹³)—why not? The sequence might go on.

2.4 In Lieu of a Conclusion

The standards parade is over. Space has been overstretched without really bringing the topic to an end. Standards are important, there are many of them, they are more or less stringent, some solid acquaintance with them helps. This may appear as a truism, but it is particularly true for mashup developers and users.

Observing standards, specifications, and guidelines that standardize/populate/underpin the web has surprised the author often enough. What standard counts, what a standard wilderness! The chapter had to tame this area to traceability in order to bring it home to its audience. If to some readers the standards parade has brought concrete hints for own system design—much the better.

Almost all readers will have objections against how the standards were chosen, left out or described in this chapter. The author shares many of them.

As long as the web keeps expanding, more and new regulation is expected to come up, and more existing standards are available for reuse. Mashup users and developers need standards, but they should ask which ones they need for their own purposes.