The CAREGIVERSPRO-MMD Platform as an Online Informational and Social Support Tool for People Living With Memory Problems and Their Carers: An Evaluation of User Engagement, Usability and Usefulness

People living with dementia or cognitive impairment (PwD) and their carers often have unmet needs for informational and social support postdiagnosis. Web-based platforms have the potential to address these needs, although few have been developed for use by both PwD and carers. The CAREGIVERSPRO-MMD platform was developed to provide both user groups with informational and peer-to-peer social support. Platform logging data were analyzed to assess the extent to which PwD (n = 37) and carers (n = 37) engaged with the platform and its social/informational features in their daily lives. Participants also provided feedback on the usefulness and usability of the platform. The majority of PwD and carers found the platform and its social/informational features useful and usable, and significant subsets of both groups utilized the platform regularly. However, carers engaged with the informational and social features to a greater extent than PwD, and users highlighted that PwD typically required regular support to use the platform.


Introduction
The majority of people diagnosed with dementia or cognitive impairment (PwD) continue to live within the community, typically supported by informal, unpaid carers (Prince et al., 2014). It is widely reported that PwD and their carers have difficulty in accessing information relating to their condition (such as behavioral, psychological, and physical symptoms, as well as support and services) postdiagnosis (Campbell et al., 2016;Lauriks et al., 2007;Ploeg et al., 2018). In addition, PwD and carers often both experience social isolation and loneliness postdiagnosis (Beeson, 2003;Spreadbury & Kipps, 2019). Web-based technologies are increasingly being seen as a cost-effective means of delivering interventions which aim to address these informational and social support needs, with the U.K. Prime Minister's Challenge on Dementia 2020 (Department of Health, 2015) calling for the translation of technological solutions into everyday practice for dementia care and support. Web-based technologies have the potential to provide PwD and carers with access to a wide range of online information, whereas online social networking and peer-to-peer support forums have the potential to address social needs by reducing isolation and loneliness. However, few technologies aimed specifically at these user groups have been developed to address these needs.
Some evidence is available regarding the efficacy of digital web-based platforms in providing informational and social support to PwD and carers, although these interventions have predominantly been developed to address the 885326J AGXXX10.1177/0733464819885326Journal of Applied GerontologyHowe et al. needs of carers more than PwD. Such resources have received positive feedback from carers with regard to providing information (e.g., Ploeg et al., 2018) and improving social connectedness (e.g., Dam et al., 2019). Only a small number of studies have investigated similar technological interventions targeted at PwD as well as carers. This may be underpinned by an assumption that PwD would not benefit from (or effectively engage with) the technology; current evidence concerning the usefulness and usability of such a tool to both user groups is currently limited and mixed. In one example, Hattink, Droes, Sikkes, Oostra, and Lemstra (2016) investigated use of the Digital Alzheimer Center (DAC), an online portal which provided information and peer support to both PwD and carers. They found that 65% of PwD and 78% of carers who reported using the DAC found it useful, particularly for accessing information which furthered their understanding of dementia, demonstrating potential for both PwD and carers to benefit from such technologies. However, Hattink et al. also found that only 45% of PwD and 53% of carers surveyed reported using the portal, pointing toward potential issues around user engagement for both groups. Another study by Killin et al. (2018) investigated PwD and carers' use of the digital support platform (DSP), an online tool providing information about dementia and community resources. They found that the DSP was predominantly used by carers, whereas PwD rarely understood or engaged with the tool. PwD's feedback suggested that they perceived the information provided on the platform to be aimed more at carers, so it is uncertain whether this lack of engagement from PwD was due to problems of usability/usefulness, or the relevance of information provided. Further evidence is required relating to PwD's engagement and interest in webbased interventions for informational and social support.
User engagement is essential to the success of any intervention. However, studies investigating the use of webbased technologies for informational and social support by PwD and carers have tended to report very limited data relating to engagement. This may be due to a lack of detailed logging mechanisms measuring participant interaction with the technology, meaning that studies have tended to assess engagement via self-report measures. For instance, Hattink et al. (2016) classified participants as active users if they reported utilizing the portal "at least twice" over an unspecified time period. A position paper by Meiland et al. (2017) on establishing the usability and effectiveness of technologies designed to support PwD and their carers highlights a need for wider reporting of logging data as a measure of the technology's effectiveness. However, there is currently very little published logging data reflecting the use of web-based informational and social support interventions by PwD and carers.
CAREGIVERSPRO-MMD is an online platform designed for use by community-dwelling PwD (with mild to moderate dementia/cognitive impairment) and their informal carers. The platform has a range of functions, primarily serving to provide informational and social support to both user groups via a social media style "News Feed." Informational articles posted on the News Feed cover a wide range of topics such as memory loss, health and social care, caring, local support groups, services, and events. In addition, the platform contains social networking features which allow participants to send messages and share information with other users. The platform has a range of other features (including a calendar/ reminder system for logging appointments and links to a range of games); these features will not be considered within the scope of this article. A more in-depth description of the informational and social features can be found in the "Method" section.
We analyzed platform logging data to assess the extent to which carers and PwD engaged with the CAREGIVERSPRO-MMD platform over a period of 6 months, both in terms of general engagement with the platform as a whole (number of visits, time spent, actions performed, etc.), and engagement specifically with features designed to provide informational and social support (predominantly within the News Feed function). To our knowledge, this is the first article to analyze logging data reflecting PwD and carers' use of such a tool in their daily lives. We also report feedback from participants regarding how useful and usable they found the platform in general, as well as the informational and social features.

Participants
Participants received access to the platform as part of a randomized controlled trial assessing the platform's impact on factors such as quality of life and carer burden (measured at baseline, 6 months and 12 months); the results presented here reflect sub-analyses from this trial. The trial recruited participants as dyads (consisting of one PwD and their unpaid, informal carer). Participants were required to be community-dwelling, with PwD requiring a self-reported diagnosis of dementia or mild cognitive impairment. Carers were required to be more than 18 years of age and self-report as the informal carer of the PwD. Both PwD and carers had to be English speaking, be able to give informed consent, and be able to (or be willing to learn to) use the technology through which the platform was delivered. These abilities were assessed at all points of contact with participants (training sessions and data collection visits). Dyads were randomized to the intervention or control group after providing written informed consent. Intervention group dyads received access to the platform, whereas control participants did not (and are subsequently not included in the analyses presented).
Participants were excluded from analyses if they withdrew from the trial before the start of the period for which logging data were analyzed, or if they withdrew before receiving access to the platform. A total of 37 dyads (N = 74 participants) were included in analysis of general platform engagement. Analyses relating to specific informational and social support features of the platform excluded "nonusers"; participants who had never or very rarely accessed the platform at all outside of training sessions (see Results for relevant criteria). Of the participants included in the general analyses, 31 (83.78%) carers were the spouse of the PwD (five were the child of the PwD, one was a friend), and 33 dyads (89.19%) lived together. Most participants were retired (85.14%), and 8.1% were unemployed. See Table 1 for additional characteristics of participants included in analyses.
All intervention group dyads were given an initial home visit in which they were provided with two Internet-enabled touch screen tablets (one for each dyad member) and trained in the use of the platform. After this, they were offered follow-up support in the form of optional group training sessions (involving tutorials in the use of various platform functions, with written step-by-step guides). During the 6 month period for which logging data were analyzed, four group training sessions were conducted per month. Of the intervention group participants included in initial analyses, 76% attended at least one group training session, whereas 64% attended multiple training sessions. Of the 51 intervention group participants who classified as "active" users, 46 (90%) attended at least one group training session, whereas 42 (82%) attended multiple training sessions. Of the 31 participants who provided usability/usefulness feedback, only two had previously attended no group training sessions.

Informational and Social Support Features on the CAREGIVERSPRO-MMD Platform
Upon logging in, users were presented with the Home Screen, which contained links to each feature. The main features relevant to the analyses presented in this article were "News Feed" and "Friends" functions.
To interact with each other (share posts and messages with other users), users had to be connected to each other as "friends." The Friends function allowed users to view the other user accounts that they were added to as friends, and send invitations to users they were not currently added to (invitations had to be accepted by the other user to connect the two accounts). By default, all users were connected to seven "professional" accounts (members of the research team) and the other member of their dyad. The platform was a closed network; all user accounts belonged to either participants or researchers working within the project.
The News Feed was the hub of information and social interaction on the platform. The News Feed consisted of posts by users, shared messages between users, and informational articles. By default, posts and messages created by users were shared with "friends," although users had the option of making posts private. Users could comment on and "like" any posts which were shared with them. Researchers regularly utilized the News Feed to share information with participants (see Figure 1 for an example); during the analyzed time period, the research team created approximately 10 posts per week on average. Some posts provided informational support (relevant to matters such as dementia/cognitive impairment, caregiving, and health), whereas others were more informal posts (relating to general topics, such as local news/events). Informational support delivered through the News Feed also included articles published via the "Content Manager" system (see Figure 2 for an example). Researchers created informational articles on a range of topics, that were perceived to be useful to PwD or carers, based on the researchers' professional experience and focus groups conducted during the early design of the platform (see Table 3 for examples of topics). Users selected which topics they were interested in receiving information about; the Content Manager system used this information to select which content to publish to their News Feed. Content Manager articles were published to each participant at a rate of approximately one article every 2 days. Publications appeared on the News Feed as a title followed by a brief excerpt. Users then had to follow a link to access the full article, which contained more detailed information and "outlinks" to external informational web pages.

Analysis of Logging Data
Participant interactions with the platform were recorded using the Matomo (2019) open-source analytics application (version 3.5.1). Actions performed within the platform (e.g., specific page visits, interactions with posts) were recorded and time stamped. It was also logged when users accessed an "outlink" to an external website, although any actions performed on external websites were not logged. Users' actions were grouped into "visits"; a new visit was started the first time a user visited the platform, and subsequently if more than 30 min had passed since the last action.  The logging data analyzed were representative of platform use over a period of 189 days. For all analyses, logging data recorded during training sessions were excluded, meaning all activity analyzed was representative of participants' independent use of the platform.

Usefulness and Usability Data
A written questionnaire assessing the extent to which participants found the platform useful and easy to use was administered during group training sessions. The questionnaire rated the extent to which participants agreed that each platform feature was useful and easy to use on a 1 to 5 Likert-type scale (strongly disagree to strongly agree). For each feature, a blank space was provided for participants to add additional feedback. Participants were also asked to rate the extent to which they considered the platform to be useful to each user group.
Usability and usefulness feedback was collected from 31 participants; 16 PwD and 15 carers. Only three of the participants who gave usability and usefulness feedback (two PwD, one carer) were later classified as non-users of the platform (see "Results" section for criteria for classifying nonusers).

Logging Data Relating to General Engagement With CAREGIVERSPRO-MMD
Visits to the platform. Over the 6 month period analyzed, participants logged a total of 7,863 visits to the platform; 2,965 (37.71%) by PwD, and 4,898 (62.29%) by carers. The median number of visits to the platform during this time period by PwD was 29 (interquartile range [IQR] = 114), whereas the median number of visits made to the platform by carers was 37 (IQR = 259).
The distribution of visits per day exhibited a strong positive skew for both PwD and carers, indicating that a relatively high proportion of users in both groups accessed the platform infrequently; 48.65% of PwD and 45.95% of carers visited less than once per week on average. However, subsets of both user groups visited the platform very regularly; six PwD (16.22% of the group) and 11 carers (29.73% of the group) visited at least once a day on average. There were seven carers who visited twice a day or more on average. See Figure 3 for histograms of the average visits to the platform per day for PwD and carers.
Mann-Whitney U tests were conducted to assess whether PwD and carers differed significantly in the number of visits they made to the platform. PwD and carers were not found to differ significantly in either total visits, U = 591, p = .312, or in average visits per day, U = 594.5, p = .330.
Prior to participating in the research, participants provided information on whether they used the Internet daily, weekly, monthly, or not at all. One-way analysis of variance (ANOVAs) revealed that total visits to the platform did not significantly differ as a function of participants' previous Internet experience for both PwD, F(3, 36) = .653, p = .587, and carers, F(3, 36) = 1.09, p = .367.
Actions performed. The mean number of actions performed per visit was calculated for each participant, and a grand mean calculated for the overall sample as well as each subgroup. Overall, participants performed a mean of 12.38 (SD = 9.69) actions per visit. The average actions performed per visit were very similar for PwD and carers; 11.87 (SD = 8.53) for PwD, and 12.92 (SD = 10.89) for carers. Mann-Whitney U tests revealed that PwD and carers did not differ significantly in either the total number of actions performed, U = 479.50, p = .119, or in average numbers of actions per visit, U = 571, p = .630.

General Participant Feedback Relating to CAREGIVERSPRO-MMD
Of the 16 PwD who gave usability feedback, 12 agreed/ strongly agreed that the platform was useful to carers, whereas 11 agreed/strongly agreed that the platform was useful to PwD. One PwD stated that the platform was "useful for people with memory problems, but they probably need help to use it." Of the 15 carers who gave usability feedback, 14 agreed/ strongly agreed that the platform was useful for carers, whereas 11 agreed/strongly agreed that it was useful for PwD. One carer stated that the platform "is useful for people with memory problems, but they need a lot of support to use the system." This view was echoed by another carer, who stated that "with help all the time, it is beneficial [for PwD]."

Logging Data Relating to Informational and Social Support Features of CAREGIVERSPRO-MMD
For analyses concerning use of the platform features relating to informational and social support, participants who never or very rarely used the platform at all were excluded so that data were representative of participants who used the platform at least semi-regularly. Participants were considered "nonusers" if they had registered an average of 0.05 visits per day (equivalent to one visit every 20 days) to the platform or fewer. After excluding participants who met this criterion, data were analyzed for n = 51 "active" participants (25 PwD, 26 carers).
Adding friends. An independent samples t test found that there was no significant difference between PwD and carers in the number of friends added over the 6 month period analyzed, t(49) = .380, p = .706. Excluding professional accounts and the other member of their dyad (to whom every user was added by default), PwD had added a mean of 19.64 (SD = 11.08) friends at the time of analysis, whereas carers had a mean of 21.12 (SD = 16.11) friends.
News Feed. Given the multipurpose nature of the News Feed (both informational and social support), logging data captured various types of interaction within this feature. First, the number of participant visits to the platform which involved accessing the News Feed at least once was considered. The number of social networking interactions made by users were also analyzed. In addition, the number of informational "Content Manager" articles and outlinks were also considered.
Visits to the News Feed. Active participants logged 3,648 visits in which the News Feed was visited at least once, making it the second most frequently accessed feature of the platform (after the "Resources" feature, which was primarily accessed for links to external web browser games). PwD logged 1,163 (31.88%) of these visits, whereas 2,485  Social networking interactions. Various types of social networking interactions within the News Feed function were logged. Two of these related to content created by the users; any posts on the News Feed which were shared with friends, and private messages sent to specific users. There were also two types of News Feed interaction which involved users reacting to the posts of others; writing a comment on a post, or "liking" a post (by tapping on a heart icon attached to that post). Descriptive statistics for these four variables are displayed in Table 2.
The majority of participants logged very few social interactions on the News Feed, with the majority of interactions accounted for by a small number of participants. Of the total 209 News Feed posts created by carers, 116 (55.50%) were created by two participants (7.69% of the group), whereas 12 carers (46.15% of the group) created two posts or fewer. Approximately 50% of PwD did not log any News Feed posts, comments, or likes, with approximately 50% these interactions recorded by PwD being logged by just two participants.
Carers engaged with all four social networking functions to a greater extent than PwD. Mann-Whitney U tests revealed nonsignificant differences between PwD and carers in the number of News Feed posts shared with friends, U = 237, p = .090, and the number of posts "liked," U = 231, p = .064. However, carers were found to comment on News Feed posts significantly more than PwD, U = 206, p = .023, and send significantly more private messages to other users than PwD, U = 200.50, p = .008.
Accessing information. The News Feed was also the primary source of informational support on the platform. This was presented through "Content Manager" articles and regular posts from members of the research team. Interaction with information on the platform was logged in two ways; accessing a Content Manager article in full (i.e., tapping the link to go beyond the brief excerpt presented directly within the News Feed), and accessing an "outlink" to an external website.
During the analyzed time period, 180 different Content Manager articles were published a total of 4,213 times to the active participants' News Feeds. Participants received a new Content Manager publication approximately once every 2 days on average (publication rates were highly similar for all participants). Participants accessed a total of 545 of these Content Manager publications (12.94% of total publications); 168 (30.83%) were accessed by PwD, whereas 377 (69.17%) were accessed by carers. Although the majority of PwD accessed very few Content Manager articles (76% accessed three articles or fewer), there were several PwD who did so regularly; three PwD (12% of the group) were responsible for 86.31% of the 168 total publications accessed by PwD (accessing approximately one publication every 3 days, on average). More carers tended to regularly access Content Manager articles; 25% of carers accessed a Content Manager publication at least once a week or more. A Mann-Whitney U test revealed that carers accessed significantly more informational Content Manager publications than PwD, U = 198.50, p = .015.
Active participants accessed 226 different outlinks to external informational websites 532 times during the analyzed time period. PwD were responsible for 182 (34.21%), whereas 350 (65.79%) were accessed by carers. A Mann-Whitney U test revealed that the difference between PwD and carers in informational outlinks opened was marginally nonsignificant, U = 224, p = .052; a result which was influenced by three PwD outliers who regularly accessed informational outlinks (accounting for 62.64% of the total outlinks opened by PwD).
To assess which topics of information were most regularly accessed by PwD and carers, each Content Manager article and informational outlink was assigned to one of 13 categories (see Table 3 for descriptions of the different categories).
As each topic of information was not represented to an equal extent on the platform, the number of different outlinks and number of Content Manager articles published within each category were taken into consideration. When accounting for this, the most regularly accessed categories of information among PwD were Legal/Financial (20 outlinks accessed 27 times, and 79 publications accessed 20 times) and Assistive Technology (16 outlinks accessed 20 times, and 145 publications accessed 21 times). For carers, the most popular topics of information were Carer Information/ Advice (19 outlinks accessed 54 times, 510 publications accessed 102 times) and Living with dementia/cognitive impairment (29 outlinks accessed 46 times, and 235 publications accessed 48 times). The least accessed categories (controlling for the number of different outlinks/publications within categories) were Social Groups and Services for PwD, and Signposting and Treatments & Therapies for carers.

Participant Feedback Relating to Informational and Social Support Features of CAREGIVERSPRO-MMD
Of the PwD who completed usability feedback, 62.5% agreed/strongly agreed that adding friends was useful, whereas 56.25% agreed/strongly agreed that this feature was easy to use. Among the carers who gave feedback, 60% agreed/strongly agreed that adding friends was useful, whereas 80% found this feature easy to use.
One PwD noted that they rarely added friends "apart from at the group [training sessions]." Two participants (one carer and one PwD) noted that they felt more comfortable when somebody else invited them to be friends, rather than sending friend invitations themselves.
The News Feed was favorably rated by the majority of participants; 81.25% of PwD and 86.67% of carers agreed or strongly agreed that they found the News Feed useful, whereas 87.5% of PwD and 93.33% of carers agreed or strongly agreed that they found the News Feed easy to use. One PwD stated that the News Feed was "the main [feature]" of the platform that they used. One carer said that they liked to "read the News Feed daily and find the information from others and the research team." Spearman's rank-order correlations revealed that the perceived usefulness and usability of the Friends and News Feed features did not correlate significantly with PwDs' or carers' prior levels of internet experience. (all p > .05)

Discussion
Results indicated that PwD and carers generally engaged with the CAREGIVERSPRO-MMD platform to a similar extent; approximately 50% of both user groups visited the website regularly (at least once per week, on average), whereas approximately a third of each group were nonusers. The majority of participants agreed that the platform was useful, although feedback from both user groups indicated that participants generally considered the platform to be more useful to carers than PwD (a comparable result to those in previous studies of web-based interventions aimed at both user groups, e.g., Hattink et al., 2016). In addition, users in both groups also indicated that they believed PwD would typically require a high level of support to utilize the platform effectively. Participants were provided with group training sessions which provided regular support in using the platform, and the majority of users who classified as "active" on the platform attended multiple group training sessions. This suggests that continued face-to-face support may be necessary for user engagement.
Interestingly, previous experience of the Internet did not appear to correlate with platform engagement or usefulness/ Social group Information on local support groups for PwD and carers.

• • Local reminiscence groups
Treatments/therapies Information relating to medication/therapies for dementia/cognitive impairment.

• • Cognitive stimulation therapy
Understanding dementia/cognitive impairment. General information on dementia or cognitive impairment.
• • Alzheimer's society-"What is Alzheimer's Disease?" guide usability feedback. The majority of PwD who claimed to use the Internet on a daily basis prior to the study were classified as nonusers, whereas 13 of the 16 PwD who claimed to never use the Internet prior to the study were classified as "active" users. This suggests that the platform was useful in providing support to people who did not routinely use the Internet. However, the majority of carers were experienced Internet users prior to the study, with more than 90% of participants' homes having access to WiFi. Therefore, this finding may not be generalizable to dyads in which carers have limited online experience. Nonetheless, results suggest that with sufficient support, PwD with limited previous online experience can effectively engage with this type of webbased platform.
The News Feed was one of the most regularly accessed features of the platform. There were no significant differences between PwD and carers in terms of visits to the News Feed, and the majority of users in both groups indicated that they found the feature useful and easy to use. In addition, both user groups tended to add similar numbers of friends. However, despite regular visits to the News Feed by active users in both groups, engagement with social interaction functions was low among both groups. Although these functions tended to be more regularly used among carers than PwD, the majority of interactions logged by both groups were recorded by a very small number of participants. Multiple users reported a hesitancy to add other users as friends, potentially highlighting that users were cautious in their online social interactions. However, this pattern of engagement is comparable with the "90-9-1" principle (Nielsen, 2006); the rule of thumb that within online communities, 90% of users will read content but rarely contribute to the network, 9% will contribute occasionally, and 1% ("superusers") will participate regularly and account for the vast majority of contributions. This pattern of engagement is observed in a wide variety of online communities, including Digital Health Social Networks (van Mierlo, 2014) and online mental health support groups (Carron-Arthur, Cunningham, & Griffiths, 2014). The logging data presented suggest that an online social network of PwD and carers may be expected to exhibit a similar pattern of engagement, and that carers may be more likely to engage as "superusers" than PwD.
Results indicated that carers used the platform to access information more regularly than PwD. The majority of PwD accessed very little information, although there were several PwD who accessed information regularly. The topics of information most accessed by PwD through the platform related to legal and financial matters (articles on matters such as Power of Attorney, benefits, and will making) and assistive technology. One of the topics least accessed by PwD was "social groups" (information relating to local support groups); this topic was regularly accessed by carers, indicating that information relating to social support in the community may be more valued by carers than PwD, or that carers typically take the initiative in respect of accessing social support. The most accessed topics of information by carers were "carer information/advice" and "living with dementia/cognitive impairment." These topics both tended to comprise relatively brief, focused articles relating to specific day-to-day aspects of living with and caring for someone living with dementia/cognitive impairment. Combined with the fact that one of the least accessed topics of information by carers was "signposting" (typically links to the home page of websites offering a broad range of information), this supports existing evidence that carers value more tailored informational support relating to specific needs, as opposed to being signposted to more generalized support (Killin et al., 2018).

Limitations
A limitation of the logging data was that it was impossible to detect whether a user had read articles which were presented in their entirety directly within the News Feed. Accordingly, it was impossible to determine whether participants had read the regular News Feed posts by members of the research team, which consisted of a combination of brief informational articles and more general and informal posts. Given that many active participants visited the News Feed regularly as well as rating it favorably in terms of usefulness despite rarely using it for social interaction (or accessing informational outlinks and Content Manager articles, in the case of PwD), it seems likely that regular posting from the research team may have been a significant factor in participants regularly accessing this feature.
It should also be noted that it was not possible to detect any instances in which dyad members may have used each other's tablets. As each member of the dyad was given a tablet and user account and instructed to only use their own, it is likely that incidents of tablet swapping within dyads were rare. However, it is plausible that there may have been some instances in which PwD's tablets were used by their carers and vice versa.

Conclusion
Overall, the results presented suggest that web-based platforms combining social and informational support have the potential to be viable postdiagnostic support tools for PwD and their carers, with logging data providing evidence that a substantial subset of both user groups are willing and able to utilize the platform regularly in their daily lives. Results suggested that PwD are able to utilize and benefit from these types of interventions (even with limited previous experience of Internet use), although may require a high level of support (such as regular training sessions and support from carers) to do so. It may be useful for future research into similar interventions to investigate whether this high level of support needs to be ongoing, or whether it can be withdrawn after some time without a significantly detrimental effect on user engagement.