1 Introduction

An estimated 55% of the global population, some 4.2billion people, live in cities and urban areas, with this expected to increase to 70% by 2050 (The World Bank 2020). Thus, the strain from urbanisation generates issues like water pollution and land degradation (Deng et al. 2019), leading to further social and environmental problems (Bibri and Krogstie 2017). Information and communication technologies (ICT) have transitioned into a mainstream debate on how to resolve these urban sustainability challenges. Digital innovation has been accepted in its’ ability to facilitate determining complex social and environmental difficulties embedded within contemporary cities (Bibri and Krogstie 2017). Studies around the smart cities concept, such as Cugurullo (2018), point out that these ‘smart’ cities tend to be unconnected and isolated. This, in turn, leads toward ‘fragmented urbanism’ and thus is far from the desired human-centred cities that we aspire to (Aina et al. 2019). The ‘smartness’ of a city thus does not guarantee sustainability (Almeida et al. 2018), with smart technology only being seen as an enabler for sustainability. However, the non-technological cities that aspire to develop sustainably also have issues with their implementation (Huovila et al. 2019); much of this stem from the ambiguity through debatable attributes alongside a complex socio-ecological structure (Molnar, Morgan, and Bell 2001). This has promoted a hybridising of the two concepts, creating the smart sustainable city, where the strengths and weaknesses of each concept can complement one another.

Smart sustainable cities are a relatively new concept, and as such, there is much ambiguity around any definition or frameworks, with these still in development. However, the idea of sustainable cities as a whole, whichever concept is adopted to achieve sustainability, is theoretically underdeveloped and implementation applicability continues to be understudied (Bibri and Krogstie 2017). At present, much of the research is towards the conceptualisation of smart sustainable cities, with a lack of empirical work to support this (Huovila et al. 2019). Despite the uncertainty around the concept, many cities globally have created distinctive visions of a smart, sustainable city.

Many of these cities globally use indicator frameworks to evaluate their progress and for decision-making; however, these indicators are generally oversimplified (ISO 2010). Thus, cities are assessed based on simplicity and ease to measure, leading to the possibility that vital elements are not evaluated as they are considered too challenging to measure. These elements can include intangible components such as citizens’ perceptions, which are too complex and subjective to be incorporated within an indicator framework leading to a further loss of the human-centred approach. However, the city is used by and lived in by the citizenry, and thus they should be incorporated within the decision-making, especially considering that sustainability is pluralistic by nature (Sharifi and Murayama 2013). The importance of engaging the citizenry is even more critical when considering each city has its’ own idiosyncratic challenges and needs, and thus solutions should be validated by the citizens (Estevez et al. 2016).

This study builds upon a prior empirical model developed by Wong and Homer (In-Press). Using the broad conceptualisation developed for the previous research, this study moves to statistically refine and validate a measure for the perceptions of a smart sustainable city. This is important as it is argued that research focused on perceptions can be more effectively utilised because individuals tend to act based on their perceptions (what they think) more reliably than objective reality (i.e., indicators) (Hansen et al. 2016). With the use of perceptions, general citizenry was asked for their impressions of smart sustainable cities, thus these may not reflect the objective evaluations of experts which would have deeper knowledge and understanding. This study intends to focus primarily on the process and methodology; thus, the preceding section gives a concise overview of the literature (for more depth, refer to [names redacted], XXXX) and recaps the empirical conceptualisation generated by [names redacted]. Following the literature review, the methodology begins by refining the statement list using exploratory factor analysis, then validating and applying the measure in a cross-national context. The discussion then not only covers the results but also looks at the implications for the methodological process. The paper then finishes with a conclusion and limitations of the study.

2 Literature review

The selected concept of smart sustainable cities refers to one of many urban forms adopted in the pursuit of sustainable urbanisation, with sustainable urbanisation composed of multiple dimensions within the confines of a city, operating social, environmental and economic elements in tridem successfully (Shmelev and Shmelev 2018). The urban form of smart sustainable city drew mainstream interest during the middle of the 2010s, generated by changing global trends around technological development, urban growth and sustainability awareness (Höjer and Wangel 2015). Other urban forms pursue sustainability, such as eco-city or compact city; however, all forms have some weaknesses, which led to an increase in the use of hybrid cities, where these forms are combined, i.e. eco-compact urban form (Bibri and Krogstie 2017). This hybridisation led to the creation of those, as mentioned earlier, smart sustainable cities, in which the concepts of smart cities are brought together with sustainable cities. However, such hybridisations are challenging to explore as the multiplicity and variations of definitions create challenges in separating the particular urban form conceptualisations (Bibri and Krogstie 2017).

Further challenges in exploring sustainable urban forms stem from the notion of sustainability itself; the ambiguity leads to debatable multifaceted, normative and philosophical attributes alongside a complex socio-ecological structure when implemented (Molnar, Morgan, and Bell, 2001). In an attempt to clarify, a sustainable urban form could be defined as a set of approaches applying the knowledge of urban sustainability and environmental technologies to the planning and design of cities (Bibri and Krogstie 2017). However, this does not assist sufficiently. When sustainable is hybridised with smart, smart cities, have a vast array of definitions with many different emphasises, with the only convergence being that of ICT (Bibri and Krogstie 2017).

The justification for the hybridisation of smart sustainable city comes from the particular forms of sustainable city and smart city facing several critiques; thus, hybridisation is an attempt at alleviating some of these critiques. These critiques include that sustainable cities that adopted the triple-bottom-line are now becoming conceptual dated as the transition to a more digitalised society is not considered. However, this digitalised society allowed the creation of multiple smart city resolutions to maximise efficiency (Huovila et al. 2019). Furthermore, the sustainable city based on the triple-bottom-line also faces criticism based on emphasising the environment and economic components, whilst social elements remain marginalised (Bouzguenda et al. 2019). This social component is particularly difficult to accommodate within measurement as intangible and subjective (Bouzguenda et al. 2019). It is also not considered an absolute, not a constant, but rather a dynamic component (Dempsey et al. 2011). The attempts to incorporate the social element through measuring ‘soft’ components, such as happiness, etc., increased the complexity rather than the intended improvement of measurement (Huovila et al. 2019). This, in turn, means evaluating the social element and tracking progress is still troublesome. This is further exacerbated by politicians and planners often making trade-offs that favour the economic aspect over the other components of the triple-bottom-line (Lorek and Spangeberg 2014). Hence creates a contradiction of protecting the environment while still maintaining a perpetually expanding economy (Martin et al. 2019).

Whilst sustainable cities have been heavily critiqued, so too has the smart city concept. The critique suggests that smart cities maintain a focus on techno-centricity whilst similar to the sustainable city, lack the humanistic social element of giving attention to the city and citizen needs (Yigitcanlar et al. 2019), stemming from the trend of being oriented toward business and professional classes (Hollands 2008). This ignorance towards the lower classes and marginalised exists within the academic literature, too; where ICT can facilitate citizen engagement, there has been little research to explore the actual practices (Granier and Kudo 2016). The critique extends by rebutting the transformative claims of the smart city, which is outsourced to large technology corporations who use the concept as ‘greenwashing’ (Hollands 2014). These large corporations pursue profits and thus emphasise the economic elements (Noy and Givoni 2018), creating a tendency to fall short of sustainability promises (Ahvenniemi et al. 2017). These critiques suggest that the preoccupation with ‘smartness’ does not guarantee the sustainability of a city, and only by further progressing the concept of sustainable urban form can the desired results be achieved (Almeida et al. 2018). Yet Hollands (2008, 2014) points out that the underdevelopment of the concept of a smart city, both theoretically and empirically, means that the critique is also underdeveloped.

These combined critiques suggest that conceptually neither a sustainable city nor a smart city could achieve sustainability by itself. The hybridisation to merge the concepts of sustainable and smart city can be achieved by using the quadruple bottom-line (Michael and Elser 2019); this would incorporate smart alongside economic, environmental and social, with the smart aspect acting as an enabler to the other components. Within this framework, the smart element is limited in enabling the social part, as previously discussed in the critiques section. Still, it can complement ecological modernisation and harmonise environmental protection and economic development (Martin et al. 2019). Thus, although smart sustainable city concept may still not contain all the solutions, it is a progressive conceptualisation.

In the approximate decade that a smart sustainable city has been in the mainstream focus, it has been conceptualised in multiple ways, reflecting upon the many interdisciplinary applications to this urban form Kremer et al. (2019). From these numerous conceptualisations, indicator frameworks have become popular for evaluating and tracking the progress of smart sustainable cities; however, although indicators are useful for a basic comparison or tracking of a single city over a longitudinal period, many have fundamental issues. Suggestions from the OECD prompt that indicators’ measurability, analytical soundness, applicability for different regions, relevance to the phenomenon and relationship to each other should be considered when selecting indicators (Deng et al. 2019). Thus, as suggested by Huovila, Bosch and Airaksinsen (2019), only six indicator frameworks can be used international however the application between global north and south and temperate and tropical locations may pose problems. These limitations of indicator frameworks suggest that a subjective measurement instrument may well assist in evaluating smart sustainable cities. Similar to the hybridisation of smart and sustainable cities, a hybridisation of using both an objective indicator measure and a subjective measurement instrument can be used in tandem. This study aims to create a subjective measure from an existing conceptualisation.

2.1 The conceptualisation of a smart sustainable city

In the interest of a holistic approach in developing a measurement instrument for Smart Sustainable Cities, the conceptualisation that has been adopted will be discussed with the methodology used to develop this. Thus, the method and results of Wong and Homer (In-Press) must be recapped to set the context upon which this study builds. That study was conducted within Bandar Sunway, just outside Malaysia’s capital of Kuala Lumpur, and self-identifies as a smart sustainable city. The selected method for this initial exploratory study was concept mapping, a structured and integrated mixed-method approach consisting of five steps; generating statements, sorting of statements, multidimensional scaling, hierarchical cluster analysis and naming clusters. This method was selected for its ability to use a participatory approach to develop a conceptual framework (Kane and Trochim 2007).

The first component of creating statements was implemented through an open-ended survey, as the approach tends to generate more honest responses and present deeper descriptions (Jackson and Trochim 2002). A simple brainstorming prompt of ‘What features would a city need for you to recognise it as a smart sustainable city?’ was used with participants consisting of those who interact with Bandar Sunway through the means of living, working or studying within the city for example. This collection of ideas was then refined to the necessary 80–100 statements using Key Word in Context (KWIC), followed by a further thematic reduction, with research agreement reducing subjectivity. This led to the 514 statements being reduced to a list of just 80.

This list of statements was then entered into The Concept System® Global MAX© browser-based analytical tool in which participants group the statements in a way that make sense to the individual. There is little guidance within this stage as it is intended to allow participants to express their cognitive relationships amongst the statements. The sorting process guidance is that; participants cannot put each statement on its own, participants cannot put all statements in one pile, and participants cannot form a miscellaneous pile. Then participants are asked to name the groups they have created, followed by rating each statement on its relevant importance in achieving a smart sustainable city. In the study, 23 participants completed this sorting exercise, with Rosas and Kane (2012) pointing out that between 20 and 30 participants are sufficient. Multidimensional scaling was conducted on the browser-based analytical tool, producing an acceptable stress value of 0.2756 (Sturrock and Rocha 2000). The cluster map for eight clusters was then generated through hierarchical cluster analysis based on an agreement of [names redacted]. Each cluster was labelled based on participant responses to represent the contained statements (refer to Table 1). The cluster rating map (Fig. 1) shows the importance of ratings as an aggregated value within the cluster. The individual statements and cluster average importance ratings can be seen in Table 1.

Table 1 Cluster Rating Table
Full size table
Fig. 1
figure 1

Cluster Rating Map

Full size image

Now that the conceptualisation has been comprehensively discussed concerning how it was developed, whilst also allowing subsequent research to adopt a similar methodology in future studies, this paper continues with the measurement instrument development within the methodology and results section.

3 Methodology and results

This paper is focused on developing a measurement instrument for smart sustainable cities. The research consists of two studies that build upon a conceptualisation of smart sustainable cities in an exploratory study using concept mapping by Wong and Homer (In-Press). Two studies are used to refine this conceptualisation and develop a measurement instrument from it, namely, Study 1: Scale items refinement using exploratory factor analysis and Study 2: Confirmation and application using confirmatory composite analysis. This is then proceeded by the previous two rounds of refinement using exploratory factor analysis and confirmatory composite analysis before finally applying and thus confirming the measurement instrument in the real-world context. This cycle of improvement ensures rigour to the process of scale development within the normative domain [importance ranking] before evaluating real-world cities in the descriptive environment to ensure the applicability of the measurement instrument. This approach was adopted to allow for a large number of statements to be used from the conceptualisation which had been developed by stakeholders initially to be used to map the phenomenon domain of smart sustainable cities. As the two studies are sequential, the methodology and results from study 1 are presented before reporting the methodology and results from study 2.

3.1 Methodology and results of study 1: exploratory factor analysis

The review paper on the topic of using concept mapping to develop measurement instruments by Rosas and Ridings (2016) was used as a guide to validating the measurement instrument. The initial step entails reducing the statement list; Rosas and Riding suggest the most common method used to reduce a statement list was to set a specific rating threshold, which for this study was to use the importance rating from the conceptualisation. With the aim of reducing the statement list of eighty by half, allowing for a fair trade-off between covering the domain and reducing participant fatigue. This was done by retaining the five most important statements from each of the eight clusters of the concept mapping, which ensured all eight domains were covered but also reduced the statement list to 40 items. An ePoster was generated advertising participation in the study for those who interact with Bandar Sunway in any form; this was placed on social media for two months. The general citizenry was recruited which may have limited knowledge compared to experts but as Hansen et al., (2016) suggest, individuals act more consistently upon their perceptions, thus making perceptions critical but not necessarily reflecting the objective realities. Participants were asked to rate each of the remaining statements on their relative importance in achieving a smart sustainable city on a scale of 1—Very Unimportant to 7—Very Important. The participants were recruited through a social media poster and were Malaysians who predominately lived or worked within the confines of Bandar Sunway (similar to the conceptualisation). Two hundred and ninety-seven respondents submitted the questionnaire, with the exploratory factor analysis performed on SPSS 25. Sampling adequacy was interrogated using Kaiser–Meyer–Olkin generating a value of 0.928 within necessary limits. A Principal Axis exploratory factor analysis with Varimax rotation was performed. The results produced an eight-factor solution; however, the results from the EFA did not align with the contents of the clusters from Wong and Homer (In-Press), with individual statements being grouped together differently. This would suggest the cognitive grouping from the conceptualisation is not statistically supported.

The statements “Smart sustainable city should have disabled-friendly facilities”, “Smart sustainable city should have integrated transportation system”, and “Smart sustainable city should have online access to public services” all had no loading above 0.400 and were thus excluded. Based on lessons learnt from Homer (2021), cross-loading factors that had both loadings above the threshold were also removed; these consisted of “Smart sustainable city should have sufficient green spaces”, “Smart sustainable city should have well-designed walking pathways”, “Smart sustainable city should be designed to be inclusive of various community needs”, “Smart sustainable city should have efficient use of energy”, “Smart sustainable city should have integrated recycling system”, Smart sustainable city should have education facilities offering world-class digital content”, “Smart sustainable city should have ease of information access” and, “Smart sustainable city should have fast internet connectivity”.

Whilst this is a considerable number of cross-loading statements, it was not surprising as many of the ideas from the conceptual domain overlapped. Thus, this large number of cross-loading items was to be expected. The number of statements that remained was still numerable, and with participant fatigue being a concern, the loading threshold for removal was increased to 0.600. This meant that “Smart sustainable city should have accessible healthcare”, “Smart sustainable city should have smart draining system to manage flash flood”, “Smart sustainable city should setup infrastructure for renewable energy”, “Smart sustainable city should have smart system to manage traffic” and, “Smart sustainable should have electric solutions for public transportation”. This resulted in an instrument consisting of 24 items, which was deemed a fair trade-off between covering the domain and reducing participant fatigue. The results from this exploratory factor analysis can be seen in Table 2, along with their assigned index number for the proceeding analytical stage. Within Table 3, the exploratory factor analysis results have been mapped back to the initial conceptualisation clusters and whilst many of the items are grouped in the same manner, there are some differences. This may well suggest discrepancies between how the initial participants cognitively relate items together through the sorting exercise and how items are statistically grouped. This may have also been affected by the many similar conceptual ideas such as Digitisation and Technology; for example, thus this may explain a large number of cross-loadings or larger [but under the threshold] loadings on multiple factors.

Table 2 Exploratory Factor Analysis Results Table
Full size table
Table 3 Mapping EFA Results onto Concept Map Clusters Table
Full size table

3.2 Methodology and results of study 2: confirmatory composite analysis

Study 2 was to assess the stability of the factors derived from study 1 and then evaluate the application of the measures. These factors’ stability was tested using Confirmatory Factor Analysis (CFA) using Structural Equational Modeling (SEM), as researchers can define and discover the vital factors and relationships which set trends in a given society (Tarka 2018). Recently Nicolas et al. (2020) also used SEM to investigate how enablers, directly and indirectly, influence the performances of smart cities deeming the method appropriate to identify latent variables of interest and establish possible causal paths among them. This, thus, reinforced the use of SEM within this study, as whilst some elements of the conceptualisation were objective, there are also many subjective and latent variables.

The CFA is being performed upon the SmartPLS software, with the Partial Least Square—Structural Equation Modelling (PLS-SEM) variation of CFA being Confirmatory Composite Analysis (CCA) as suggested by Schuberth et al. (2018). The choice to use PLS was made as it has been advised that researchers should particularly use PLS-SEM with CCA in the case of measurement models that indirectly measure conceptual composites (Hair and Sarstedt 2019). This is because Hair and Sarstedt (2019) state that composite-based SEM methods, such as PLS-SEM, use total variance to develop linear combinations of indicators to form composite variables that empirically represent the conceptual variables. Whereas factor-based SEM methods empirically represent the conceptual variable using common factors consisting of only common variance that explains the covariation between their associated indicators. The CCA procedure was outlined by Hair et al. (2020). The CCA process includes seven steps; 1. Estimate of loadings and significance, 2. Indicator reliability, 3. Composite reliability, 4. Average Variance Extracted (AVE), 5. Discriminant Validity—HTMT, 6. Nomological validity and, 7. Predictive validity. Whilst this study follows these steps and subsequently reports them, steps number 6 and 7 were outside the scope of the current project; however, an additional step was introduced in the form of known group validity to fill this gap.

The study aimed to consist of between 180 and 200 participants from Malaysia and a similar number from the UK, thus beginning to test the cross-cultural applicability of the derived measure with a total sample size of between 360 and 400. Participants were asked to rate the 24 items derived in study 1 upon the relevant importance of representing the normative typology. Within the initial concept map and study 1, eight clusters were derived; however, with the substantial number of cross-loadings experienced, there is an expectation that a refinement of the composites may be necessary. Additional validation was conducted with Known Group Validity, which involves the instruments’ ability to differentiate among groups. The groups were expected to rate differently on specific traits or aspects (Netemeyer et al. 2003). In this case, the relevant importance between Malaysian and UK on the components of smart sustainable cities is analysed by a t-test on a composite score (Rosas and Ridings 2016).

Participants for the Malaysian samples were gathered through social media posts using an ePoster and a link to the survey, with 177 participants recruited in this manner. The UK participants were recruited through the third-party survey website; Prolific, where the study details were displayed so participants would only participate if they felt they had sufficient knowledge to answer the questions, with the sample consisting of 182. This then gave a combined total of 359 participants. Both groups of participants were formed through the general citizenry, which as prior discussed may have limited specialised knowledge but are more likely to act upon their perceptions of a smart sustainable city. Participants were asked to rate the relative importance of each of the 24 items in achieving a smart sustainable city on a scale of 1- very unimportant to 7—very important. The results from the confirmatory composite analysis can be seen in Table 4 and, as expected, had to be substantially refined to ensure an acceptable fit. Whilst many items presented cross-loadings and were moved to form larger composites to improve the fit of the model, only four items had to be removed because they cross-loaded between multiple composites. This removal was below the 20% threshold for removal of items that are recommended and included; SSC1—smart sustainable city should have clean air, SSC9—smart sustainable city should facilitate work-life balance, SSC10—smart sustainable city should be designed to encourage active lifestyle and, SSC19—smart sustainable city should encourage the use of hybrid cars. With the removal of these items, the results within Table 4 show a good fit with construct reliability and validity being above the accepted values of Cronbach’s alpha (α) > 0.700, rho_A > 0.700, Composite Reliability (CR) > 0.700 and Average Variance Extracted (AVE) > 0.500. However, whilst discriminate validity was acceptable, three values were borderline (in bold and italics in Table 4) due to the ambiguity about the cut-off point; Kline (2011) suggests 0.85, and Gold et al. (2001) suggest 0.90. Surprisingly, the factors’ loadings were substantially high for an initial scale development study. Hulland (1999) indicates that loadings above 0.600 can be acceptable within exploratory analyses, but this was unnecessary. As the stability of the composites had now been confirmed, they were now named based on the underlying theory and themes: Planning, Environment, Social and Smart.

Table 4 Confirmatory Composite Analysis Importance Results
Full size table

Next, an independent 2-tailed T-test was conducted between a composite average score for each factor of the measurement instrument. Within the Planning composite, participants from Malaysia had a mean (M) of 6.148 and a Standard Deviation (SD) of 1.520, whilst the participants from the UK had an M of 6.453 and an SD of 0.628 with a significant difference in mean scores, t(357) = − 2.498, p = 0.013. Within Environment composite, participants from Malaysia (M = 5.629, SD = 1.463) compared to participants from the UK (M = 5.120, SD = 1.085) with a significant difference in mean scores, t(357) = 3.752, p =  < 0.001. Within Social composite, participants from Malaysia (M = 5.863, SD = 1.429) compared to participants from the UK (M = 5.456, SD = 1.014) with a significant difference in mean scores, t(357) = 2.913, p = 0.004. Within Smart composite, participants from Malaysia (M = 5.681, SD = 1.368) compared participants from the UK (M = 5.659, SD = 0.882) with an insignificant difference in mean scores, t(357) = 0.177, p = 0.860. These results would suggest that the measurement instrument is validated through known group validity, as three of the four factors have significant differences between Malaysian and UK Gen Z, whilst one factor demonstrates there is a similarity as there is an insignificant difference in the average importance rating of the factors referring to Smart Sustainable Cities.

The study now moved to test the measurement instrument in the descriptive typology. It is applied to specific city locations, effectively moving from the normative or desired smart sustainable cities to the real-world evaluation. Greco et al. (2019) suggest that a robustness analysis should follow the construction of an index, as it provides a quality assurance tool and overall transparency. Whilst the robustness analysis was not conducted, the shift in typologies (normative to descriptive) and the application of the measurement instrument in several contexts do test its applicability. This consisted of asking the same sample of Malaysian and UK, which evaluated two cities known for sustainability. The selection was based upon cities which are prominent, and the participants would most likely be familiar with the amenities and thus can give an accurate evaluation; for Malaysian participants, this was Melaka and Putrajaya, whilst for the UK participants, this included Edinburgh and London. For readers unfamiliar with Malaysia, Melaka is the capital of the state of Malacca, with a population of 579,000 and is one of the cleanest cities in South East Asia and is the oldest Malaysian city on the Straits of Malacca. While Putrajaya is a planned capital city, functioning as Malaysia’s administrative and judicial capital, with the seat of the federal government shifting to Putrajaya in 1999 from Kuala Lumpur because of overcrowding and congestion, development began in August 1995 at an estimated cost of US$8.1 billion. Thus, with each participant from the prior sample evaluating two cities, the data set doubled in size to 718 measures. In Table 5, the results of the confirmatory composite analysis can be seen within the descriptive typology. The construct reliability and validity being above the accepted values of Cronbach’s alpha (α) > 0.700, rho_A > 0.700, Composite Reliability (CR) > 0.700 and Average Variance Extracted (AVE) > 0.500. What should be noted is that although some of the loadings are lower (but still above acceptable levels) than the importance value, the discriminate validity had improved with all values below the lower cut-off of 0.85 suggested by Kline (2011).

Table 5 Confirmatory Composite Analysis Evaluative Results
Full size table

Proceeding the confirmatory composite analysis within the descriptive typology, another known group validity was tested. This time, the individual cities had their averages of each factor compared to look for differences. Having more than two groups meant that an ANOVA had to be used rather than a T-test, as Homer (2021) used. The results from the ANOVA can be seen in Table 6, with all four factors demonstrating significant differences; this was to be expected as the four cities have very different dynamics and thus adds further validity to the measurement instrument. A Bonferroni post-hoc was conducted to investigate further the differences in the means of the cities that the participants had evaluated; this can be seen in Table 7. The post-hoc table further strengthens the validation. It demonstrates that some cities have similarities as not every comparison is significant; only those in the ‘Mean Difference (I-J)’ column in bold are significant. Of the 48 comparisons, 36 are significantly different. The conceptualisation has been comprehensively interrogated to produce a measurement instrument designed for citizens to evaluate the ‘smartness’ and ‘sustainability’ of their cities. This paper now continues the discussion to look at the implications of this study.

Table 6 ANOVA Results Table
Full size table
Table 7 ANOVA post-hoc Table
Full size table

4 Discussion

The paper will proceed with the discussion; however, the meaning of individual clusters from the conceptualisation will not be discussed as this can be referred to in Wong and Homer (In-Press). The discussion will revolve around what the additional two studies have uncovered and added to the original study before narrowing the discussion to specific theoretical implications and methodological implications.

The measurement instrument developed from this study composes of;

4.1 Planning

  • Smart sustainable city should have disaster resilient design

  • Smart sustainable city should have green buildings

  • Smart sustainable city should have climate resilient infrastructures

  • Smart sustainable city should have well distributed housing plan

  • Smart sustainable city should be an integrated township

  • Smart sustainable city should have purposed build and well-designed buildings

4.2 Environment

  • Smart sustainable city should have a clean environment

  • Smart sustainable city should have water management

  • Smart sustainable city should have integrated energy management

  • Smart sustainable city should have clean water

  • Smart sustainable city should have efficient waste management

  • Smart sustainable city should have efficient waste water management

  • Smart sustainable city should minimise waste generation

4.3 Social

  • Smart sustainable city should have real-time transportation information

  • Smart sustainable city should have instantaneous reach to authorities

  • Smart sustainable city should have a complete range of security system from preventive to reactive

4.4 Smart

  • Smart sustainable city should have a thriving Research and Development (RandD) community

  • Smart sustainable city should have Internet of Things (IoT) enabled infrastructure.

  • Smart sustainable city should have ICT infrastructure

  • Smart sustainable city should have advance technological integration

4.4.1 Theoretical implications

The measurement instrument consists of four elements, namely; Planning, Environment, Social and, Smart. The instrument does not follow the quadruple bottom line of economic, environmental, social, and the additional smart element. This adoption of the quadruple is in-line with many other fields of sustainability studies, where the quadruple bottom line proposes that the triple bottom line is not enough and an additional, more specific factor is needed to achieve sustainability (Michael and Elser 2019). It could be debated that the Planning aspect could be compared to the Economic element in so much that effective planning would increase efficiency and, in turn, reduce the costs, thus demonstrating an economic component to effective planning. The other elements align well with their corresponding features; the environment has a theme around waste and water and energy to protect these resources and use them efficiently, indirectly preserving the environment. Social aspects revolve around safety and security whilst also including transportation, whilst the smart component focuses upon integrating technology into the city’s infrastructure. The proposition here is that the sustainability aspect is not isolated; rather, it takes all four elements of planning, environment, social and smart to achieve a sustainable city. This would align with the literature as ‘smartness’ does not guarantee sustainability (Almeida et al. 2018); thus, smartness is an enabler to achieve the desired sustainability.

Whilst planning was included within the conceptualisation and the scale development from this study, the spatial dimension of the city did not materialise. In particular the spatial component of poverty, which Stretesky et al., (2004) determine as a structural characteristic of cities and has a relationship to the level of violent crime rates. The conceptualisation’s context may be partially responsible for this but also that smart sustainable cities selected; Melaka, Putrajaya, London and Edinburgh, whilst have impoverished areas, may well be considered affluent areas and thus justifying the investment in the smart technology. This creates a noteworthy area for discussion around whether there is a poverty discrepancy between cities adopting a smart sustainable approach and those which do not? It is noted that the spatial concentration of impoverished areas creates a “social-structural milieu” that works to prevent members of a community from creating and maintaining the basic institutional structures that prevent social problems such as crime, i.e., residents of impoverished and socially isolated communities are likely to have less access to police resources (Stretesky et al. 2004). This may then lead to the prioritisation of embedding these institutional structures over the implementation of smart sustainable infrastructure. Further to this, Liborio et al. (2020) suggest intra-urban inequality, or other phenomena, can be interesting to determine what is the influence of spatial dependence in defining the scores of the areas of a specific phenomenon. This would promote that using the developed Smart Sustainable City instrument may vary across a single city; this intra-urban inequality may well create a substantial impact on the measurement instrument depending upon the area of the city where its’ citizenry completes the evaluation.

The terminology of smart sustainable city presents an issue in itself; whilst academics and, to a lesser extent, practitioners have clearly defined distinctions between the various urban forms, i.e. compact-city or smart-city, the general populous of these cities are not so well informed. The theoretical implications are that when working on a bottom-up paradigm, the different urban forms lose their distinctions, and we see a convergence of concepts. While the citizenry can benefit most from a smart sustainable city, the terminology used to address the implementation is not ‘user friendly’. Whilst the study focused upon current cities, which are generally conceived as having smart and sustainable aspects already, the question arises what if a city was starting its journey towards sustainability from a low level, would the citizenry understand the measure? Thus, there may be a necessity to explain why. Why do we need a thriving RandD community? Why do we need climate resilient infrastructure? Hence, as the theoretical and implementation of sustainable cities develops, the dissemination and education for sustainable cities need to build also.

4.4.2 Methodological implications

The methodological process for this study was composed of a rigorous process of refinement, with the initial research composing of an exploratory factor analysis from the most important statements of the broad conceptualisation of a smart sustainable city. This reduced the large list of items down to a representative set of items numbered 24 items across eight factors; however, numerous items were conceptually similar and were likely to produce cross-loadings. The subsequent stage first sought to validate the measurement instrument within the normative [importance] domain. With the removal of 4 items and the reconfiguration to 4 composites, we found a good fit for the model. An instrument of 20 items was a good balance between covering the domain and balancing participant fatigue.

Further validation was confirmed with known group validity between Malaysian and UK participants. Finally, the domain was switched, so the participants evaluated two well-known cities in their respective countries; hence the instrument was used within the descriptive domain. The measurement instrument had a good fit and was again validated with known group validity, but this time between the different cities with unique characteristics. This methodological approach of an iterative refinement cycle can be used for scale development from a large number of items in many circumstances.

What can also be noted is how whilst the conceptualisation adopted to develop the measurement instrument appears to be theoretically and cognitively sensible, once the statistical verification process begins, this breaks down somewhat. This may be because, within the normative [importance] domain, many of the items are considered very important. Thus, attempting to differentiate between these prove difficult statistically and explains the high number of cross-loading. The operationalised measure instrument varies widely from the original eight cluster conceptualisation; this can be seen in Table 8. The most notable variation is that none of the statements from the Community-friendly Township cluster was maintained in the final iteration of the measurement instrument, despite five statements being included at the exploratory factor analysis stage.

Table 8 Composites Mapping to Conceptualisation Clusters
Full size table

Whilst this study’s scale development is based upon a subjective evaluation of the citizenry; as individuals tend to act more reliably on their perceived reality rather than objective reality (Hansen et al. 2016), to encompass a holistic measurement approach, an objective indicator system could be used in tandem. Composite Indicators have aroused the interest of researchers from the most varied areas and have been addressed from different perspectives that aim to capture the multidimensionality of the phenomena (Libório et al. 2020). This can relieve some of the subjectivity of the measurement instrument developed in this study as experts can assign the importance of each criterion relative to the others (Greco et al. 2019), thus creating objective indicators with the aid of the citizenry importance ratings and their knowledge in the field.

An important incorporation within this paper is within study 2, where the typology is switched from the normative to the descriptive. The use of stakeholder theory, as citizens are stakeholders, allows this switch of typologies using Donaldson and Preston (1995) three typologies of stakeholder theory; instrumental, normative and descriptive. Freeman (1999) defines these clearly; descriptive typology is how the world really is, normative typology prescribes how the world should be, whilst instrumental typology links means and ends. With Trevino and Weaver (1999) questioning whether there is an empirical stakeholder theory (descriptive or instrumental) to integrate with normative theory, how things should be are usually quite different from the way the world really is. This questioning by Trevino and Weaver is addressed by this switching of typologies, as the measurement instrument is developed in the normative typology of how citizens think the smart sustainable city should, before then switching to descriptive typology to evaluate how these cities are. This means that the measurement instrument can track the explanatory progress of the city towards the desired normative smart sustainable city, which the citizenry wants.

To highlight, another methodological practice used within this study is known as group validity; this practice evaluated how the measurement scale can distinguish between groups of individuals, and in this case, cities, that are expected to score differently on specific attributes (Netemeyer et al. 2003). This study adopted known group validity within the final study, but for both measures undertaken, the instrument’s normative [importance] typology was compared between Malaysians and United Kingdom participants. The second part of this study contrasted different cities within the descriptive typology, with each participant evaluating two cities from their respective countries with other characteristics. This known group validity only consisted of a small number of comparisons that could have been made, as various different demographics could have been incorporated or different cities that both groups of participants should be familiar with, i.e. New York or Paris. The implications for the methodology are that known group validity can be incorporated easily into many studies with minimal forward-thinking and can give extra depth to the validation process. Furthermore, known group validity should not be used as a simple ‘add-on’ to the methodology. Still, it should be used strategically and holistically to construct a network of comparisons like how nomological networks are constructed.

5 Conclusion

How smart sustainable cities can be evaluated is a persistent problem. Although there are many objective indicators, they are fraught with difficulties and fail to incorporate subjective elements. Adding to this is the sustainability concept’s ‘constructive ambiguity’, both helping and hindering. This leads to the evaluation of smart sustainable cities needing both objective and subjective measures, are the objective indicators tend to be frequently replete with naivety or overly simplified. This paper developed a measurement instrument based upon a prior conceptualisation that embraced the subjective nature of the citizenry’s perceptions of a smart sustainable city. The measurement instrument was initially refined from a large statement list of 80 from the initial conceptualisation before statistically honing this instrument through multiple stages of statistical analysis before applying it in the real-world context in various cities in Malaysia and UK. A twenty-item measurement instrument consisting of four factors, Planning, Environment, Social and Smart, was developed from this study. These results support current theoretical perspectives with only minor variations from the core theory; however, this better reflects the dynamics of the smart sustainable city phenomenon. The measurement instrument should be used to complement the existing objective smart sustainable city indicator frameworks, which, when used together, can create a holistic evaluation of smart sustainable cities.

5.1 Limitations

This study represents only a limited number of the 193 United Nations (UN) member countries which have a vast and wide-ranging set of climate variations, economic development and cultural influences which may affect the interpretation of what a smart sustainable city is. Limitations from the study may originate from the conceptualisation used to develop the measurement instrument. The concept map by Wong and Homer (In-Press) focused upon a very narrow group of individuals in a geographically specific area. Although the study then generalises the results through cross-country validation, it may well be that the second country may have conceptualised a smart sustainable city differently in the beginning. However, with such a broad use of statements (eighty) in the conceptualisation, there would likely be similarities between conceptualisations. Additionally, only participants from Malaysia and the UK were used, whilst this may create a good environment to test the measurement instrument as they were expected to contrast sharply.

The study also presents a limitation in how individuals were recruited, for the Malaysian sample only social media was used and for the UK sample only Prolific. Thus, this may create issues with the sample composition as only those with either social media or a Prolific account had the option to partake with the study and may create an underlying bias within the results. This a problem to be faced especially when considering larger cities, such as the more than 30 mega cities with over 10 million populations, collecting a representative sample is a huge challenge. It was outside the scope and means of this study to collect such specific samples. This limitation leaves a research gap in that the comparison between demographics of age, education as well as others, may not be adequately covered and the opportunity to compare and evaluate the differences between the various demographics is lost. This study can be deemed an initial foray into developing a measurement instrument, however to confirm or refute the usage of the instrument developed, there is a need for validations with representative samples within multiple contexts.

5.2 Future research

This study has only evaluated a narrow part of the field of study; future research may wish to proceed with conducting a global conceptualisation of a smart sustainable city and develop a measurement instrument before applying it to various contexts around the planet to validate its applicableness. This does create a challenge, though, as tensions between citizenry may arise, as many factors such as economic development, climatic zone and culture may all affect the citizens’ priorities.