Keywords

1 Introduction

Norway has legislated that all public ICT-solutions must be inclusive and universal designed, in order to combat democratic, economical and ethical issues in the expansive self-service eSociety. In addition to determine quality assuring criteria for final solutions, best-practice process criteria could help ensure universal design (UD). British Standard 8878 suggests a user-centred approach to web accessibility [1]. UD processes reported in literature are often user centered, inclusive and iterative. However, even if universal design is viewed as an extension of user-centered approaches there are multiple practiced degrees of user contact, user involvement, user sensitivity and critical thinking in the field [2, 3]. The range of user-centeredness and epistemological beliefs varies [4]. This study is a step towards understanding the impact of methodology on universal design quality. The goal of this exploratory study is to gain more insight into how universal design is practiced by Norwegian experts on universal design of ICT. Through survey research the paper explores methodologies strategies, epistemological views and paradigmic stances currently employed.

2 Background

Academic approaches for universal design of IT are often user-centered and seem well aligned with ISO 9241-210 [5]. Common methodologies are user-centered design (UCD), inclusive design, user sensitive inclusive design, emphatic design, user involvement and participatory design [4]. At least two epistemological cultures appear to be present in the field, with different philosophical justifications for what one believes is valid knowledge and how it can be aquired. The first epistemological culture seems focused on technology, checklists, standards, automatic tests and inspections [6, 7], to view UD as an added constraint and influenced by (post)positivism [8]. The second epistemology is positive towards UD, focus on users and includes epistemological reflections linked to critical, interventionistic and interpretive paradigms. The second culture is frequently identified in participatory design (PD), inclusive design (ID) and user-sensitive inclusive design (USID) approaches [9, 10].

Methodological approaches seem related to paradigms, overarching mindsets including academic culture, ideology, epistemologies and worldviews. Different strategies may be applied when faced with real-life challenges such as tight deadlines and limiting resources. One is rationally defining the best way forward within constraints; top-down stepwise and/or empirical, experimental and inductive. Such strategies connects to mechanical worldview [8] viewing software engineering as a complex problem to be solved through analytical approach, and hard systems thinking where a solution may be considered correct or not-correct based on fulfilling specifications. It appears such views may be tied to a user-centered methodology using low user contact and focused on end-result [4].

A second strategy is to actively influence decisions, including altering specifications and constraints. This is an interventionistic strategy, tied to a dialectic approach where several possible solutions and viewpoints are considered [8]. The expert is again taking on the role of a problem solver, but this time in a more political manner. The typical interventionsitic stand is technology is non-neutral. The focus is not so much on creating a correct solution in relation to specifications, as arriving at a good solution from ethical and socio-technical viewpoints. Such stances seem common within participatory design and user sensitive inclusive design. A third approach is focusing on dialogue in order to come to a mutual agreement, where the goal is to arrive at a solution that fits all stakeholders to the largest extent possible. Such romantical views fit with soft system thinking which encourages considerations of different perspectives and negotiations. An interpretive (or constructivist) paradigm could be tied to romantical views, opposing the positivist emphasis of facts being facts, instead viewing subjective “truths” as individual explanations of empirical experiences (relativism).

3 Research Approach

The study focuses on Norwegian domain experts developing universally designed IT-solutions. Conducting an in-depth interview study prior to a survey in order to better frame the items and focus of a survey was considered. This article focus on five items designed to map epistemologies and methodology. In addition, the survey covers specific methods usage and practices, user groups in focus and map key definitions through a total of 21 items (including background data). The study is approved by the Norwegian Social Science Data Services (NSD).

3.1 Target Group

The target group is identified using a non-probabilistic draw from a not easily well-defined population on the basis of established basic data [11]. A list of domain experts was comprised using the following approach; (1) experts among the members of the recent Norwegian network focusing on Universal Design and ICT, (2) experts among companies sponsoring Oslo Interaction Design Association and (3) experts being referred by identified experts. The experts are selected for their visibility in the field over perceived academic background and area of expertice. The goal was to indentify 30–50 domain experts. The resulting list of survey recipients included 70 experts from 13 enterprises; 8 consultancies, 2 medium-sized companies, 2 smaller R&D companies, 1 non-profit institute and 1 government agency. At least 13 of these are well-known for their expertise. An estimate is that around 20–40 % have a high domain expertise while the majority of around 30–50 % have a fairly high competence. The final sample is considered sufficient for seeking insights over generalizable results, representing major enterprises and institutions in the field.

3.2 Survey Design

In order to explore how user-centered and empathic the experts are the survey asks for degrees of agreement with different strategies. Experts are asked for agreement with the approaches: (A) user-centeredness without direct user contact, (B) user-centeredness with direct user contact (specifying the goal of the user-contact is to deepen knowledge), (C) user-centeredness emphasizing user-involvement (specifying the expert participates in users everyday life), (D) participation design (states users are equal to designers/developers and asked for input throughout the process - typical traits in participatory design) and (E) empathic design (without tying understanding needs to involvement or contact, exemplified as simulating disability or observing use). It was assumed that strategies A to D would be on a scale with increasing level of user contact, from no contact in A to high-contact and involvement in D. The user-centered and empathic strategies are phrased in relation to “my work”, thus linked to what the experts do and not necessarily what their opinions are. While this may introduce a bias, but it also makes makes the question less provoking and easier to answer. For agreement measures, a 4-point Likert scale was used, forcing the respondents to take a non-neutral stand. The same was also done for exploring agreement with polarized mechanical, romantic and interventionist worldviews and interpretive or positivistic paradigms.

From a theoretical point of view, a negative correlation is expected between mechanical and romantic views, and between interventionistic and loyal beliefs. The experts are further asked about agreement with 6 paradigmic statements. The 1st, 3rd and 5th statements are assumed positivistic and stress the importance of objectiveness, factual findings and precice criteria. The 2nd, 4th and 6th are interpretive and emphasize mutual understanding, collaboration and end-user involvement. The respondents are asked to select whether they prefer qualitative methods providing closeness to users, in-depth understanding and rich insights, or quantitative methods providing overview, validity and representative information. The phrasing of both options are positive and attempted equated for desirability. The relativist stand is phrased as viewing truths as interpreted and constructed based on interpretations, common perspective needs to be agreed upon. The opposing stance is uncovering truths through triangulation, counteracting erroneous perspectives.

4 Results

4.1 Survey Sample

26 experts (37 %) responded to the survey. Only complete survey responses were allowed. 39 % of the respondents are women and 61 % men, which is considered an equal distribution to the ratio of 37 % women and 63 % men in the target group. Years of experience ranged from 2 to 25 years, with aritmethic mean 7.73 and median 7. This is considered high compared to the age distribution of a majority below 40 years of age. The impression is that many highly experienced experts are responding to the survey. No biases are identified in the respondents compared to the sample. Job titles vary greatly, with 21 unique titles across the 26 respondents. Only 2 titles specify universal design as area of expertice. Most experts report they are interdisciplinary and 73 % work within three areas or more including at least two of the common areas visual design (65 %), programming (65 %), interaction design (85 %) and content production (50 %). 46 % of the respondents have interdisciplinary academic backgrounds, ranging from pedagogics, law and journalism to more traditional development and design disciplines.

4.2 Methodological Strategies

Table 1 shows that experts are fairly divided on a no-contact strategy (A) as well as participatory design (D), and seem somewhat partial to high-contact involvement (C). However, they seem to agree with some-contact strategies to understand user needs (73 % for B and 88 % for E). Table 2 shows there are strong correlations between strategies C and D, as well as between disagreeing with A and agreeing with strategies C and D. As expected, strategy E does not significantly correlate with the other strategies, however neither does B. There are no identified background variables influencing strategy responses.

Table 1. Agreement with methodological design strategies
Full size table
Table 2. Correlations between design strategies (Spearman’s rho, 2-sided)
Full size table

4.3 Epistemological Stances and Worldviews

61.5 % prefer qualitative methods while 38.5 % prefer quantitative. 27 % align themselves with relativism. Table 3 present the frequencies of agreement with mechanical, romantic and interventionistic views. There is a weak tendency between ethical views, which could indicate a boosting underlying factor (e.g. ethical awareness). There are moderate correlations between romantic and ethical views, but none between mechanical and romantical stances (Table 4). In order to investigate further how the views are distributing themselves in relation to each other and whether they are measuring different views or not, a Principal Component Analysis (PCA) was conducted. PCA was conducted in a purely exploratory fashion, as the data are not continuous or nearly continuous. The findings are interesting: 2 factors can be found, which together accounts for 76 % of the total variance. This indicates that two different worldviews are measured in the item. Those agreeing with mechanical views are the only ones loading Component 2 on 0,945, in addition to loading component 1 on 0,188. The other views are all loading highly in component 1. In other words, this item seems to measure mechanical versus non-mechanical views.

Table 3. Agreement with worldview
Full size table
Table 4. Crosstabulations between worldviews (Spearman’s rho, 2-tailed)
Full size table

4.4 Paradigm Stances

There is a strong correlation between the assumed interpretive 4th and 6th statements on collaboration and involving end-users (\(\rho \mathrm{s}=0.66\)), but none to the 2nd on mutual understanding (Table 5). There are moderate correlations between objectiveness and precise criteria (\(\rho \mathrm{s}=0.0.53\)) and precise criteria and finding facts (\(\rho \mathrm{s}=0.47\)). An exploratory PCA show 3 quite clear components that explain 79 % of variance. The statements load the first two components inconsistingly, but the 2nd statement on mutual understanding is the only strongly loading the last component. Looking only at the assumed positivist statements, one component is identified. Looking only at the three assumed interpretive statements, two different components are found, where the 2nd loads Component 1 on 0,189 and Component 2 on 0,975, while the other two load Component 1 on 0,912 and 0,884 and Component 2 on 0,04 and \(-0,25\).

Table 5. Agreement with paradigmic stances
Full size table

4.5 Methodological Traditions

A moderate correlation (\(\rho \mathrm{s}=0.4\)) is identified between mechanical views and agreement with a no-contact strategy. Highly significant correlations are found between the 6th stance on end-user involvement and user-involved strategies (strong for C \(\rho \mathrm{s}=0.67\), moderate for D \(\rho \mathrm{s}=0.42\)). Mechanical views correlate negatively to user involvement (\(\rho \mathrm{s}=-0.41\)) and 6th stance (\(\rho \mathrm{s}=-0.55\)), as do the 1st stance on objectiveness to romantical views (\(\rho \mathrm{s}=-0.50\)). The impression is of a connection between mechanical views and a no-contact strategy on one hand, and non-mechanical alignment and user-involved methodologies on the other. Preferring qualitative methods correlates negatively to mechanical (\(\rho \mathrm{s}=-0.48\)), no-contact (\(\rho \mathrm{s}=-0.45\)) and 1st (\(\rho \mathrm{s}=-0.49\)) statements. A crosstabulation shows none preferring qualitative methods fully agree with no-contact strategy A. A majority of the sample agree with strategy E, which correlates moderately to romantical (\(\rho \mathrm{s}=0.41\)) and loyal (\(\rho \mathrm{s}=0.43\)) views. Non-relativist epistemologies correlates moderately to 3rd stance on factual uncovering (\(\rho \mathrm{s}=0.44\)).

5 Discussion

Overall, the sample seems aligned with user-centred principles [5]. There are indications of opposing strategies, but expected correlations between epistemological beliefs and methodological approaches are blurred. Experts seem fairly consistent in expressing adherence to high-contact user-involved strategies versus a no-contact approach throughout survey items. Measuring alignment with mechanical versus no-mechanical views is successful, but separating romantic and interventionistic views is not. The assumed positivistic 1st, 3rd and 5th statements likely measure similar stances, while the 2nd measure something different than the 4th and 6th. It is unknown whether literature based theory is correct in pitting mechanical views against intertwined romantism and interventionism, and positivist against intertwined interpretive and critical. An alternative hypothesis would be that romantical views correlate more with interpretive traditions focused on facilitation, interpretation and cooperation, while e.g. USID and PD belong to critical design and interventionist non-relativistic stances.

There seems to be an acquiescence response related to epistemological views or tacit knowledge instead of conscious epistemological stances, as experts agree with both romantic (89 %), mechanical (85 %) and interventionistic (74 %) views. A redesign could combat effects of items designed equated for desirability. Correlations indicate strategies C and D imply a user-involved style (assumed high-contact) while strategy A measure an opposing no-contact (or low-contact) strategy. Strategy B may be a social desirability bias. Strong relationships between stances on collaboration and end-user involvement may be influenced by the fact that both are highly immersed principles.

A crosstabulating on relativism and quantitative-qualitative preferral shows that the most common category is to prefer qualitative methods and use method triangulation. This indicates pragmatic experts utilizing a wide range of strategies from different epistemological traditions. There are no clear relationship between the background of the experts and their epistemological beliefs. This is not suprising due to the interdisciplinarity of the experts and the low N of the sample. The experts work across disciplines and have backgrounds from several fields, with different individual combinations. The population “experts within universal design of ICTs” is not easily defined. Ensuring validity through sampling is challenging. Filtering items ensuring the experts are in the target population may continue to be necessary. Self-reported expertice in the area may help establish data validity. One could consider adding self-measurement of depth of expertice.

6 Conclusion

There are indications of diverging methodological styles in the Norwegian sample between a mechanical no-contact tradition and user-involved approaches. Some connections are identified between epistemology and methodological alignment, but more varied than theory would suggest. This could be indicative of interdisciplinarity and pragmatism, point to tacit knowledge and aquiescence effects or suggest that theoretical assumptions do not fit real-life practices. Links between methodology and overarching paradigms are still blurred. Compared to investigating epistemology and methodology though literature, this domain expert survey bring new insights that future studies will continue to explore. In addition to improved survey items, an in-depth interview study focused on UD success projects will investigate methodology, constraints, epistemology and interdisciplinarity. Continued work hopes to contribute to exploring relationships between epistemologies, methodologies and UD quality.