Accessible Blaise 5 Surveys: Challenges and Solutions
Karen Brenner and Karen Moyes, Westat
Test results from our Blaise 4.8 surveys turned up both intentional and inadvertent accessibility issues. For example, the ACASI questionnaire purposely did not allow keyboard use, to prevent respondents from finding ways to access system settings or even survey data. With the conversion to Blaise 5, we investigated the challenges of implementing accessibility. We wanted developers to understand the techniques and importance of questionnaires that are fully compliant to conformance criteria: to not only reduce maintenance costs, but also increase the number of respondents who can use the surveys.
First, we created an Accessibility Training for Developers. Second, we developed Best Practices for Blaise. With these two approaches, teams can build in accessibility during design and development, saving time, effort, and remediation costs. In this session, we will share information about our training, best practices, and results of addressing Section 508/WCAG 2.0 conformance criteria in Blaise 5.
DIM-Device Instrument Manager
Max Malhotra, University of Michigan - Survey Research Center
Using Blaise 5 for interviewer-administrated surveys pose unique technical challenges, some of these technical challenges include running offline distributed surveys. To address these challenges we developed an in-house application called Device Instrument Manager (DIM).
We required a mechanism that was robust where an interviewer could synchronize their laptop with a central repository and receive the components that are pertinent to them and be confident they could conduct offline interviews in a seamless manner and once their tasks were completed and they had internet access they can synchronize their data back to the main repository.
Prior to the development of DIM, we did not have a mechanism in place that could be coupled with an in house sample management system to provide a succinct user experience and to address the delivery and subsequent data collection which worked for both single-mode or mixed-mode environments.
This paper will discuss two major components of the DIM and some of the subcommands this application consist of such as:
1. Blaise Sync:
a. Check Version (i. Download Instruments, ii. Perform Data Model Migrations)
b. Upload Cases
c. Download Cases
2. Run Survey
a. Run Pre App
b. Start Survey
c. Run Post App
d. Pull Values
This paper will delve into how and why the above commands were required and will talk briefly about each command's implementation strategy, as well as go into our future plans for utilizing the write and download interceptors more extensively. We will also touch on some of the trials and tribulations we faced along the way until we got to a successful outcome.
A Case Management App (CMA) for mobile/tablet devices
Gina-Qian Cheung, University of Michigan and Lon Hofman, Statistics Netherlands
Now Blaise 5 App (on Android and iOS) has been developed, and we can load Blaise 5 instruments on smartphones and tablets for interviewers to conduct interviews.
However, we need to have a Case Management App (CMA) to do:
Delivery of sample lines to interviewers.
Installation and management of samples within interviewers’ devices.
Launch Blaise 5 app instruments for data collection.
Administration of case statuses (record appointments, enter call notes, etc…).
Sending survey data to the central database.
Assembling one data file from interviewer’s data files on the server-side.
During the presentation, we will demo functions on the mobile/tablet device and also the central office management system.
Blaise 5 in SHARE
Maurice Martens, UVT
The Survey of Health, Ageing and Retirement in Europe (SHARE) is a multidisciplinary and cross-national panel database of micro data on health, socio-economic status and social and family networks of about 140,000 individuals aged 50 or older (around 380,000 interviews). SHARE covers 27 European countries and Israel in 40 locales and is currently fielding its ninth wave. The questionnaire is fielded in CAPI mode.
CentERdata scripts the questionnaire in Blaise and has developed several tools that connect to these questionnaires. For latest wave the questionnaire was migrated from Blaise version 4.8 to version 5.4.
This migration was not only a change within the Blaise environment but also forced an adaptation of all tools we developed that interfaced with Blaise in our architecture. This step meant a new case management system was developed, new interfaces with our Translation Management Tool (TMT) and a new design for the Data Delivery backend were created.
This paper discusses these solutions and will also address some positive and some negative findings. The Blaise 5 development environment, full Unicode compatibility, the introduction of roles were greatly appreciated. The limited functionality for triggering events, keyboard navigation and control over lookup tables is something that should be improved on. We will discuss the workarounds we implemented to get these features in.
Blaise 5 CATI implementation at Statistics Netherlands (CBS)
Bryan Bungardt, Statistics Netherlands
Statistics Netherlands is in full swing with the introduction of Blaise 5. We are currently working on CATI going live may 2020. I will share with you our thought on how we are using CATI and how it looks.
The Awakening – Using Blaise 4 for Alternative Solutions
Bryan Bungardt, Statistics Netherlands
For years the Blaise team at the Census Bureau has been using Blaise capabilities conservatively, refraining from some of the not well-known solutions offered by the software. As years progressed and the need of alternative solutions grew, the team began to experiment with things that we may not have otherwise considered. While experimenting with these capabilities late in the life for Blaise 4, we wonder how these capabilities will transition to Blaise 5 and how they will affect the way we use it.
This paper will focus in the following Blaise 4 capabilities:
• The use of directives. We have never used directives for any of our surveys; however, the need for them became obvious while migrating our Software Configuration management solution to GitHub.
• Additional uses of Manipula, which we feel is the best tool in the Blaise software.
- As part of the Data collection tool – possibly using Manipula/Maniplus to replace our current Event History Calendar written in C#.
Continued Adventures Transitioning from Blaise 5 to 5
- As part of our help display – possibly using Manipula as an alternative solution to display help items.
- As part of our output processing - creating a pseudo customized Manipula ASCII relational output script that fits in our current output extraction process.
Vito Wagner, Peter Kilpatrick and Charles Less, US Department of Agriculture - National Agriculture Statistics Service
Although much of the individual survey code translates easily with the included conversion tools, many pitfalls await your transition. Other papers have documented numerous coding modifications needed to ones datamodel code and this issue will continue to plague the developer. Developing a house style layout gives your organization many possibilities, but with great potential come great responsibilities. Additionally, not all layout features in Blaise 4 are easily paralleled in Blaise 5. Changing versions within Blaise 5 also poses unique challenges. Major releases and minor releases sometimes change the columns required resulting in incompatibility of prior projects within the same table structure of a new project. These errors manifest themselves in multiple messages which can be frustrating to the user and difficult to describe when seeking assistance. Internal organizational process may also require changes with some being very subtle and easily overlooked such as changes in the order that the Blaise system executes some request. We look to lead the reader through some of these struggles and share in the victories that will come.
PAPI to Blaise 5 – Challenges and Solutions in Creating Complex Tables
Emily Caron, RTI International
Recent Blaise 5 development at RTI International included creating large tables to display preload data, with the option of entering or editing the data values in similar fashion to what was previously done on paper. This brought about challenges in coding and configuring the Resource Database. The process we followed and the solutions we developed to overcome the challenges will be discussed in the paper using a household roster example.
Using the Resource Database to control web security
GJ Boris Allan and Peter Stegehuis, Westat
When mounting most surveys, it is common for web interviews (WIs) to be hosted by a management system (MS). The MS organizes necessary data and then runs a chosen WI. The MS is programmed in some language such as C#, in combination with HTML and similar, and we assume the WI is programmed in Blaise 5. The MS controls web data-security for the WI, and our aim is to program the WI without any need to take into account the MS and associated concerns such as web security of the WI. By adding a standardized single parallel block that does not affect WI program code, and by adding the necessary intelligence to standardized expressions in the WI resource database (the .blrd), the WI can be made secure without modifying the basic Blaise 5 program code.
Using the Resource Database to adapt to session timeouts
GJ Boris Allan, Joseph Allen and Siu San, Westat
If an interview session times out, an obvious method of recovery is to use an error page to trap the error and then launch a call to a recovery URL. Unfortunately, because we have lost contact with the session data, it is problematic to recover the case ID of the interview we just lost (State.KeyValue is null). Also we have to hardcode the value of the recovery URL in the error page because any value stored in the survey database is unavailable as we have lost contact with the session data. Using our resource-database methodology developed for web security we can handle session timeouts and do not have to hardcode URLs – we can modify the URL without changing the package, because we just change the preloaded data.
Using Expressions in Blaise 5
Nikki Brown and Yelena Beale, Westat
This paper will examine ways in which the Expression Editor in Blaise 5 can be used to link specific layout objects to templates in order to create a layout instruction. Expressions created in Blaise can be very straightforward or incredibly complex, each providing powerful capabilities to the survey designer. Expressions are created by utilizing the drop-down lists with functions and variables each editor provides in its current context. We describe our experience learning to use the different Expression Editors available in Blaise 5 to meet client requirements.
User experience: Master/Detail alternative
Bryan Bungardt, Statistics Netherlands
Since the arrival of Blaise 5, Statistics Netherlands has been looking for a way to implement the ‘Master/detail’ solution we had created in Blaise 4. This feature was used for a large amount of our business statistics that required a large amount of repetition of blocks. With new features that have come available, we can now transfer the last of our business statistics to Blaise 5.
Sustainable Questionnaire Development with Colectica and Blaise
Jeremy Iverson and Dan Smith, Westat
Blaise Colectica Questionnaires is a survey specification creator, and now integrates with the new Colectica Question Bank. This new tool allows users to author questions and sequences of questions with a simple, web-based interface. Users can tag questions with categories, add custom metadata, indicate how response data appears in resulting datasets, and see surveys in which questions have been used. All changes and usages of questions are tracked in the metadata repository for a full audit trail. Documentation about the resulting data can reference the source questions to provide full lifecycle view of the data lineage. Blaise Colectica Questionnaires can connect to Colectica Question Bank or Colectica Repository to use single questions or full sequences of questions within a survey instrument.
Using Blaise Colectica Questionnaires, surveys are designed once and can be published as PDF specifications, paper forms, Blaise 5 data capture instruments, and standardized XML. The tool allows rapid, iterative survey development and testing. Survey authors can collaborate within the tool, making comments and edits that are tracked in a metadata repository, instead of exchanging Word documents or text files by email.
In the past year, Blaise Colectica Questionnaires has also added support for in-survey search, survey validation, intelligent drag-and-drop survey organization, visual calculation editing, external data lookup, and custom Blaise code in expressions and rules.
Structured Questionnaire specification process for Blaise 4 and challenges in its transition to Blaise 5
Martin Zelenύ, Czech Statistical Office
Blaise software has been in use at the Czech Statistical Office already since 1997. After its start in the national Labour Force Survey, it is now used for all household surveys. With the growing number of surveys implemented, including the move to a modular survey infrastructure with integrated surveys, there was a growing need for questionnaire specification management and structured communication between survey methodologists in charge of individual survey modules, and the (rather small) core team of Blaise programmers. A dedicated software tool was developed for structured specification of questionnaires for survey content methodologists, supplemented with conversion „robot“ module for automatic generating of Blaise code include files for survey modules. With the transition to Blaise 5 under way, new challenges occur for the existing questionnaire specification system, including some more strategic decisions as to how to move forward and adjust to the rich Blaise 5 environment. The paper will briefly describe the questionnaire specification and management process in place, and subsequently address the existing challenges connected to the Blaise 5 transition.
Future proof questionnaire design for social research: challenges and opportunities of a smartphone first approach
Jeldrik Bakker and Vivian Meertens, Statistics Netherlands
The coverage and use of mobile devices have increased very strongly over the last five years and are anticipated to grow for the coming years. This change in communication has already led to an increasing percentage of respondents that start online surveys on mobile devices, despite the fact that the surveys are often not fit for small screens and mobile device navigation. National statistical institutes in charge of creating European Social Surveys (ESS) have begun to react to this trend and started to explore smartphone options. A general research question that needs to be answered; What is needed to keep our ESS fit for the future? Several research projects on ESS surveys are described with respect to fitness criteria, challenges and opportunities to apply a smartphone first questionnaire design on European Social Surveys. Lessons learned can be shared and to answer research and implementation questions.
Blaise 5 CAPI in collaboration with COTS software
Rogier Hellenbrand, Statistics Netherlands
Statistics Netherlands is in full swing with the introduction of Blaise 5 on different modi: CAWI has been implemented several years ago, for CATI we have conducted several pilots in production and will be going live (meaning the survey will be used to produce statistics) may 2020. For CAPI we have done several Prove of Concepts and we have started the implementation of Blaise 5 for the actual interviewing process on (possible offline) mobile devices, in collaboration with a Dutch Commercial of-the-Shelf software product called Link2.
This product takes care of the distribution of CAPI cases between the available interviewers, as well as registration and accountability of the individual visits to respondents’ homes (including time registration for the direct hours within the company’s ERP system).
During this presentation I will give you a demonstration of Blaise CMA and Link2. Furthermore I will explain the interfacing we envisioned between both systems. As you will be aware: synchronization is a possible pitfall when working with multiple systems in the same domain.
The Blaise 5 Multimode Management System
Mark M Pierzchala, MMP Survey Services, LLC
The Blaise 5 Multimode Management System (MMS) will be demonstrated and discussed. The MMS architecture closely follows the design agreed by the BCLUB Multimode Management Group as demonstrated at IBUC 2018. The MMS was first prototyped in Blaise 4 and an information webinar was given in August 2019. It was ported to Blaise 5 in November 2019 where work has continued.
While the overall architecture follows the BCLUB design, there were many details to work out. In addition, the MMS is designed to fit into an institute's existing infrastructure. For example, most institutes already have a Sample Management System (SMS). Such a system can be used to send mail and email, and to receipt responses when they arrive, among many other tasks. The Blaise 5 Multimode Management System does not replicate such functionality.
The Blaise 5 MMS does provide a Survey Handling System (SHS) with a Survey Handling Database (SHD). The Survey Handling Database keeps track of survey management statuses, counts, and indicators. It does not contain any Personally Identifiable Information (PII). Such confidential data are held in the institute's own Sample Management System.
The Survey Handling System issues instructions for all survey management actions. To do so, it must be able to communicate with an institute's systems. Further, the Blaise 5 MMS should operate the way the institute wants. This is done through specification databases. There are databases for (1) survey design parameters, (2) happenings, and (3) actions. It also works with Blaise 5 CATI. The MMS should be able to produce reports using an institute's own coding system. Further, there are hooks in the system to implement management responsive design.
How all this is elegantly done will be revealed at IBUC 2020 in Cyprus.
Towards a modern mixed-mode Labour Force Survey
Trond Bεsnus, Statistics Norway
The Labour Force Survey (LFS) at Statistics Norway is currently a CATI-only survey running in Blaise 4.8. Statistics Norway will phase out this version of Blaise in 2021 at the latest, which makes it necessary to convert the survey to Blaise 5. New EU regulations harmonizing European social statistics will also lead to changes in the LFS questionnaire. These two changes represent an opportunity to rewrite the questionnaire to support mixed mode data collection, but also to simplify the administration of the survey which currently is quite labour intensive. A pilot for a mixed-mode LFS survey was conducted in 2018, and the valuable lessons learned will be an important input for the development work.
This paper describes briefly the current LFS-survey and the mixed mode LFS-pilot but will mainly concentrate on the changes planned for the new questionnaire. The goals for the new questionnaire are: 1. offer a secure CAWI mode to the respondents, 2. a simpler and better structured Blaise questionnaire which should be easier to maintain, 3. more efficient data collection and simpler administration of the survey. We will also discuss changes to our case management system to facilitate mixed mode surveys, and other additions such as two factor authentication for CAWI-surveys.
Field Properties Values: A Tool to Identify and Adjust Missing Data from 'Relation' Extraction
Mohammad Mushtaq and April Beaulι, University of Michigan
The Panel Study of Income Dynamics (PSID) has been using Blaise since 2003 and “ASCII-Relational” data export option to output data into SAS files. During the early stage of Blaise 5 development, the PSID applications development team has used reverse engineering to transform data from “wide” to “relational” format for “All Stars” pilot. Later, PSID staff worked closely with Blaise 5 development team to create tool to export Blaise 5 data into “relational” format, tested and provided feedback to Stats Netherlands during the development phase. The tool is now integrated with Blaise 5 Control Center and being used by the PSID in two surveys.
In a mixed-mode survey, it’s important to identify the source and type of missing data values. The web instrument does not allow “Don’t Know” and “Refused” whereas such values are allowed in CATI interviews. In web, the values could be missing due to “On-Route” (but not answered by the web respondent) vs. “Off-Route” (not presented to web respondent). Since the data from mixed-mode survey is delivered in a combined Blaise 5 data file, therefore, it’s important to harmonize missing data values across both sources. This is done by using Field Properties Values file.
It was observed from pilot studies that Field Properties Values from “wide” are larger than that of “relational” extraction.
In this paper, we examine Field Properties Values from “wide” and “relational” Blaise 5 data extraction, identify missing observation and/or values from “relational” files. Develop methodology to complete the “relational” files where question was on-route but not answered by the respondent of web survey. This distinction is important for data processing team of the PSID and will be used to write explanation for missing values in the public release files. Also feedback to Statistics Netherlands about Blaise 5 “relational” export and possible improvement.
Using Field Properties to Assist Editors
Peter Kilpatrick and Charles Less, US Department of Agriculture - National Agriculture Statistics Service
By declaring Field Properties of remarks, original values, and overlay values in our Blaise 5 instruments, we are able to store supplemental Field level values and use those stored values for multiple purposes that include quality control and data analysis. Being a statistical agency, we are keen to assist our analysts with the necessary information and tools to ensure that our data are accurate. Using Field Properties effectively allows us additional layout options and area to store data without adding additional programming burdens for our developers of individual projects.
Measuring housing and living conditions through respondent photos
Ralph Dolmans, CBS, Goran Ilic and Peter Lugtig, Utrecht University, Maarten Streefkerk, Tilburg University and Barry Schouten, CBS and Utrecht University
In various surveys, such as housing surveys and surveys on living conditions (e.g. EU-SILC), respondents are asked to report on characteristics of their houses. Characteristics may consist of the general state of the dwelling, the energy use by the household and measures they took to reduce energy use, the types of devices and equipment they have, the sizes and volumes of rooms, the size of the garden and the proportion of the garden area that has pavement. These characteristics often concern information that respondents do not know and/or find hard to report. As a consequence, statistics based on such data are subject to measurement errors.
A possible solution to assist respondents and to reduce measurement error is to offer respondents the option to make photos with their mobile devices and submit these instead. Photos, or summaries of the photos, are received by the survey institute and processed. Processing consists of image preprocessing, text mining and/or image recognition machine learning algorithms. Construction and implementation of a fully automated processing pipeline is not trivial, and respondents may still need to be involved in checking outputs.
In this paper, we present a first study where photos are made by respondents from within a Blaise questionnaire. The experimental survey is linked to the Dutch Housing survey and conducted in the LISS panel of CentERdata, Tilburg University, in November 2019. Around 2000 panel members were invited to participate and produce photos on three topics. We discuss the results of this study and the technical performance of the questionnaire.
Data Collection Management Systems in Statistics Finland
Pyry Keinonen, Statistics Finland
This paper discusses how Data Collection Management System was developed in Statistics Finland and in which way it is used in practice. Data Collection Management System was developed in 2017 to 2019 and was implemented to production in January 2019. System uses Blaise 5 as primary Data Collection System.
Data Collection Management System in Statistics Finland consists of the primary data collection management service and from two smaller sub-services. The main service provides tools for survey, sample and case management and monitoring of data collection. One of the sub-services provides the user interface for case and interview management for interviewers, and one provides the infrastructure and API for web-survey data collection. Together these services provide the necessary functions for a multi-mode data collection process.
Statistics Finland has had a need to develop a data collection management system because the old operating environment has a lot of manual work stages that require effort. These workflows are largely built around the operating models of the Blaise 4 production environment. The development of the Data Collection Management System has enabled production of reports on data collection and comprehensive management of multi-mode data collection. Most importantly, it enables online and offline interviewer data collection at the same time as online data collection.
The Data Collection Management System is built to utilize Blaise 5 as a data collection tool for online and interview data collection. Therefore, following the Blaise 5 Evolution path is crucial because the effects are directly reflected in the definition and implementation of the system development needs.
Future development work may include implementation of the Blaise 5 CATI system and other data collection systems. In addition, the development of utilizing geographic data and information into the system to allocate cases to interviewers and reduce logistic costs is underway. In the future, the data collection management system will be used for all personal data collection and the Blaise 4 production environment is being driven down.
Blaise 5 Scaling Experience
Mangal Subramanian, Kathleen O'Reagan, Arthur Menis and Ray Snowden, Westat
This paper will discuss the various technical and operational challenges faced during development, configuration and deployment of Blaise 5 web for a large scale project with a sample size of over 300,000. We will examine the decisions taken along the way, challenges faced, lessons learnt and recommend best practices for handling large scale Blaise 5 web projects. We will compare and contrast the configuration that we ended up using for our production environment vs a true distributed Blaise 5 server park based set up. We will also briefly discuss some new technology options that can be considered for future implementations that can simplify configuration and deployment of Blaise 5 projects.
Managing a complex scenario for the collection of business report forms (work title)
Leif Bochis Madsen, Statistics Denmark
The latest two years Statistics Denmark has been working on the replacement of Infopath forms in our Business Survey Division.
Now, six questionnaires are used in production, while six others are under development and should be ready for production by the end of 2019. Still, almost sixty questionnaires should be converted to Blaise 5 within the next three years.
The paper describes the status of the project including a number of issues that have been addressed and some of the tools we are building in order to incorporate the use of Blaise in the existing environment for collecting business form reports.
Also, the paper describes our work on managing a range of servers needed in the environment and our considerations on automatic generation of questionnaires.
The efficacy of a respondent centric approach - evidence from online first random probability survey in the UK
Joe Herson, ONS-Office for National Statistics, UK
The UK government has a Digital by Default strategy which means that by 2020 digital self-service is the default option for people who can use it, not the only option. Coupled with public expectations to be able to do surveys online, and increasing use of smartphones to perform online tasks means the Office for National Statistics (ONS) has invested in a transformation programme that focuses on research to deliver and integrate respondent centered online data collection. We have taken a ‘blank page’ approach to the redesign of our respondent journey.
ONS practices respondent centrism in its approach to design; we place the respondent at the forefront of our design process by aligning the questions and flows with their mental models. By doing so we create a questionnaire that collects accurate data which meets analytical requirements whilst also delivering a positive respondent experience.
This talk will share findings from a multi-wave online only random probability survey – the Labour Market Survey (LMS), which used a prototype labour market questionnaire and sampled 50,000 households across G.B. Households were given a 10-day period to complete online, and were sampled for subsequent waves upon completion to measure attrition in an online mode.
The talk will reflect on the following:
- Variability of online take-up rates across sampled areas
- Did the respondent centric approach appear to help reduce attrition?
- Sample composition, compared with the current LFS
- Can methods be employed to reduce attrition?
Laura Wilson, ONS-Office for National Statistics, UK
The UK government has a Digital by Default strategy which means that by 2020 digital self-service is the default option for people who can use it, not the only option. Coupled with public expectations to be able to do surveys online and increasing use of smartphones to perform online tasks means the Office for National Statistics (ONS) has invested in a transformation programme that focuses on research to deliver and integrate respondent centered online data collection. We have taken a ‘blank page’ approach to the redesign of our respondent journey which has been met with success.
This talk will share our approach to achieving this and the principles that the social survey research and design team use to develop online-first mixed-mode surveys. ONS practices respondent centrism in its approach to design; we place the respondent at the forefront of our design process by aligning the questions and flows with their mental models. We create a questionnaire that collects accurate data which meets the requirements whilst also delivering a positive respondent experience which is vital for voluntary longitudinal surveys.
This talk will provide tangible examples of changes to questions, including tips for other researchers to take away and apply. It will also discuss novel techniques such as combined cognitive and usability testing and why it is essential for successful delivery and good data quality in self-complete modes.
We optimise for the mode and design ‘smartphone first’. This talk will demonstrate through evidence how and why this approach is critical for success. By optimizing for smartphones, we create cleaner designs, ensure higher quality data and reduce the break-off rate for respondents who are unlikely to return on another device or in another mode.
We will share our practical examples and recommendations on techniques, questions and approaches.
Evolution of Blaise Survey development in Statistics Finland
Pyry Keinonen, Statistics Finland
This paper discusses how Blaise Survey Development is organized in Statistics Finland and how the Survey Development process has evolved during Blaise5 implementation from 2017 to 2019. Statistics Finland has implemented Data Collection Management System which has influenced and set requirements for Survey Development. Data Collection Management System has been in production since January 2019.
This transition included change in work methods and culture. The renewed ways to work have caused a need to recreate the in-house survey development process and recognize the roles needed in daily basis work. Instead of starting Survey Development from scratch and creating customized surveys on demand, the focus was to find new ways to reduce the amount of effort by standardizing. The aim was to streamline Blaise Survey Development.
Standardization has helped both the transition process from Blaise4 to Blaise5 and creating new surveys from scratch. This meant in practice the implementation of the use of version control, standardized survey-independent-layout, and standardized survey-independent base code as a part of survey development. This way we have been able to harmonize the development phases of Blaise5 questionnaires and reduce the effects caused by person dependent practices.
The next step is to implement Agile methods in survey development starting from 2020. This step includes the processes and implementation how to lead and organize our Survey Development and work resources as a whole. The goal is to reduce wasted effort and make resources available efficiently when needed.
The power of Blaise Analytics: An example of predicting break-off behaviour
Jeldrik Bakker, Statistics Netherlands
It has been a long standing dream to answer the big questions of survey research: "What is a good and what is a bad survey question?", "How good or bad are the answers from a respondent?", and "What is a good survey and what is a bad survey?". Due to various limitations, it has until now been difficult, if not impossible to answer these questions. We used a data driven approach to first find all data, which might be relevant. Secondly we defined over 100 different indicators, consisting of respondent characteristics, page & question characteristics, and survey characteristics, which should help answer most survey related questions. Finally we used these indicators to explain survey behaviour and try to answer the big questions. During this presentation we will show an example of what is possible with these methods to explain why people break-off in surveys. We explored the answers and the answer-behaviour of over 100.000 respondents across multiple surveys. For this, we analysed more than 35 million records of paradata, collected by the Blaise software, and combined this with survey data and metadata. Some of the results were in line with previous findings, but some findings were new and unexpected. At the end we will present our take on how to continue with these tools/methods.
The co-existence of Blaise 4 and Blaise 5 in CYSTAT
Niki Chrysostomou, Charoulla Charalambous, Constantinos Mina, Charalambos Charalambous and Costas Diamantides, CYSTAT
CYSTAT has a long experience in the use of Blaise. In the recent years, Blaise 5 has been introduced and CYSTAT is now in the situation of using both versions 4 and 5. Blaise 5 is currently used in two surveys, the ICT usage in enterprises and e-commerce (ICT-ENT) and in Employment and Job Vacancy.
The ICT-ENT has been carried out annually since 2004. In 2009, the CAPI mode was implemented in Blaise. In 2015, a web questionnaire was developed for the first time in Blaise 5. At the same time, the questionnaire used for the personal interview is developed in Blaise 4. For the 2020 survey, the CAPI will be implemented in Blaise 5 on a pilot basis.
Employment and Job Vacancy is a quarterly survey in which the data is collected through telephone interviews. Since 2016 the Blaise 4 CATI module has been applied for data collection and management of the survey. In the framework of the implementation of CYSTAT’s strategy to offer respondents alternative means to provide the required information, the CAWI option built on Blaise 5 was implemented in 2018.
Statistics on Income and Living Conditions (EU-SILC) is an annual cross-sectional and longitudinal sample survey and is the main source for data and indicators on income, poverty, social exclusion and living conditions in the European Union. EU-SILC is carried out in Cyprus since 2005 and the data collection incorporates multi modes such as personal interviewing, telephone interviewing and administrative registers. CAPI is currently implemented in Blaise 4 and the aim is to use multi mode data collection with Blaise 5.
The aim of the paper is to present the experiences of CYSTAT in multi mode data collection by using both versions 4 and 5 and to share the future plans in using Blaise 5.
Converting a Multi-Lingual CAPI questionnaire from Blaise 4.8 to Blaise 5.
Andy Wastson, Office for National Statistics UK
The National Survey for Wales (NSW) is an income backed survey conducted on behalf of the Welsh Government by Office for National Statistics (ONS). The survey is run over 12 months and involves around 12,000 face-to-face interviews covering a wide variety of topics such as; Child online safety, local environmental quality, diet, physical punishment of children and satisfaction with Welsh Government. The data collected through this survey are processed by a research team and prepared for delivery to Welsh Government.
NSW is conducted in English and Welsh and was the first entirely multi-lingual surveys conducted by ONS. According to the Annual Population Survey (June 2019), approximately 30% of people in Wales can speak Welsh. As such, Welsh Government requires that the NSW questionnaire is available entirely in Welsh, should a respondent wish to respond to the interview in Welsh.
The NSW is currently conducted using Blaise 4.8, utilising the “multiple languages” data-model properties. It also incorporates the UTF-8-character encoding, for any special Welsh characters that are not in the English alphabet. In addition, Welsh Government have very specific requirements for the presentation style of the questionnaire, resulting in several user defined Fonts and Infopanes being created in a unique Mode Library for the survey.
Converting a multi-lingual questionnaire from Blaise 4 to 5 has raised some challenges. For example, incorporating the unique survey requirements, the pre-existing corporately themed templates from the resource database and using standardised text roles within the questionnaire has been thought provoking. However, it has also highlighted some of the features within the Blaise 5 conversion tool set that have made this task significantly easier. And when previews of the new layouts were sent to Welsh Government, they were extremely well received and they provided positive and encouraging feedback.
Moving the International Passenger Survey Questionnaire to Blaise 5
Steve Maurice, Office for National Statistics UK
The International Passenger Survey (IPS) is a continuous survey that has been running since 1961 by the Office for National Statistics (ONS). This is the main source of information in the UK on international travel and tourism, as well as the associated expenditure.
Between 700,000 and 800,000 face-to-face interviews are conducted every year at the major air, sea and tunnel routes to and from the UK by a national team of 240 interviewers.
Having recently moved from paper collection to Blaise 4.8 collection using Tablets (as described at IBUC 2016), ONS has now made the decision to move ALL Blaise 4.8 surveys to Blaise 5.
We decided that the move from Blaise 4.8 to Blaise 5, and the move from our in-house case management system to DEPApp, would be a good opportunity to merge the IPS ‘Shift Details’ and ‘Main Collection’ questionnaires. The goal of this was to make processes easier for interviewers and back at the office, as well as reducing the number of files being delivered/stored.
This paper details the experience of moving the IPS questionnaire from Blaise 4.8 to Blaise 5 while merging the two IPS questionnaires, some of the challenges we encountered along the way, and how we overcame them.
The value of modern-day technology (Microsoft Azure Cloud) - Lessons learned in testing the integration of Blaise 5 to MS Azure
Lynley Speers and Graeme Simpson, Stats NZ
Stats NZ is in the process of transitioning Blaise 4.8 to Blaise 5. Using Blaise 5 means that the current back-end functionalities we have used to deploy Blaise 4.8 survey questionnaires will completely change.
Household surveys are currently completed through face-to-face with our field interviewers. Subsequent interviewers for the Household Labour Force Survey (HLFS), quarters 2 to 8 are usually done by phone by our Contact Centre. Surveys are completed off-line using a laptop or a desktop. Blaise 4.8 is used to create customised off-line questionnaires for Stats NZ's household surveys. The offline Blaise 4.8 questionnaire is uploaded to laptops of field or desktops of Contact Centre staff. These are delivered via Lotus Notes. Replication is used to upload the completed interviews in the IDE Load Area.
Stats NZ is testing Blaise 5 such that respondents will be able to complete their surveys online in addition to the current CAPI and CATI. To do so, Blaise 5 must integrate with our current IT systems (e.g. the use of Microsoft Azure Cloud, Salesforce customer relationship, Trak case management, EPIC platform) and should align with the DevOps model of our Digital Business Services. Stats NZ has endeavoured to build the automation of environments using Infrastructure as Code (IaC) to rapidly deploy solutions that are able to communicate to cloud solutions and the onshore enterprise solutions. This includes deploying Extract Transform Load (ETL) tools into the cloud in order to make the overall solution flexible and scalable to other surveys in the future.
Our presentation also includes the challenges to secure the privacy and confidentiality of survey respondents’ data in near-shore Australia cloud, maintenance of our social license as a trustworthy organisation in protecting respondents’ data, value-for-money of onshore vs off-shore cloud storage and integrating Blaise 5 with our current IT systems.
Migrating a Listing Instrument to Blaise 5
Michael K. Mangiapane, U.S. Census Bureau
The Survey of Construction Listing Instrument (SOC LI) is a unique CAPI survey since it collects a large list of building permits before a complex sampling script runs in the instrument to select a number of permits for further follow-up. When the SOC LI was coded in Blaise 4, it took advantage of advanced features such as Interchange and being able to call a Maniplus script as an event or from the menu. Without them, it would have been extremely difficult to implement the functionality required for the instrument.
With Blaise 5 we are interested in learning if we will still have the required functionality (e.g. unduplication, sampling) and how we can update and improve the SOC LI for potential use as a Blaise 5 instrument. This paper will present the progress made on moving the SOC LI to Blaise 5 and what we have learned so far. It will also document the challenges encountered while migrating the instrument and the solutions developed.
Spot the difference. Automated visual testing with Blaisium
Angelo Pascale, Arthur van den Berg and Joris Bleus, Statistics Netherlands
With each new Blaise release, not only should the functionality of existing Blaise web questionnaires remain the same, they should also still look the same. It takes a lot of time to test this manually, that’s why we want to test this automatically.
There are no affordable, easy-to-use tools in the market that can do this for all operating systems and devices. We have built a Python framework that can do this ‘Blaisium’.
With Blaisium, one can test web questionnaires on different devices and device farms. At the Blaise test team we have linked Blaisium to Browserstack. In this way we can test the visual aspects of web questionnaires on Windows, macOS, Android and iOS devices.