Skip to content
Draft
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 40 additions & 1 deletion 01b-specific_data_types.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -74,4 +74,43 @@ For more information see:

## Synthetic Data

## Summary
Synthetic data are artificially generated datasets designed to reflect the structure and patterns of real-world populations, without containing information about actual patients. (https://pmc.ncbi.nlm.nih.gov/articles/PMC11555762/). There are many ways to generate synthetic data. Some approaches are simple, like creating partially synthetic data by replacing potentially identifying pieces of information with synthetic values. Others are more complex, like using AI systems to fully generate synthetic patient notes. The goal of synthetic data is to mimic the characteristics and patterns in real patient data, while avoiding privacy concerns, cost, or biases of using real data sets.

### Uses of synthetic data

There are several possible uses for synthetic data:

- Testing or validation: synthetic data may be easier to create or access than real clinical data, while being similar enough to test or validate new tools, methods, or data handling pipelines. It can also be shared between institutions, and used to compare the performance of competing tools.
- Augmenting or balancing data sets: real patient data can have subgroups with low representation, which can lead to algorithmic bias in some applications. Synthetic data can be used to balance data so that there are more observations that represent these groups.
- Supplementing controls groups in clinical trials: Conducting clinical trials requires a lot of time and money. Synthetic data has been suggested as a supplement to control arms in clinical trials, in order to reduce costs or realign funds to add more patients to the experimental arm (https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0000581).
- Augmenting data for rare disease areas: Rare disease research often suffers from small sample sizes of patient data and can be especially susceptible to privacy concerns, which can lead to underpowered studies. Synthetic data has the potential to be used for detection and development of treatment for rare diseases (https://pmc.ncbi.nlm.nih.gov/articles/PMC11958975/)

### Resources for generating synthetic data

There are a wide range of ways to generate synthetic data for different uses. Some tools to access or generate synthetic clinical data include:

- [Synthea](https://github.com/synthetichealth/synthea): a synthetic patient population simulator, that generates synthetic medical histories. This tool can be used without restriction for research, industry, or government.
- [SyntheticMass](https://synthea.mitre.org): a synthetic dataset of residents of Massachusetts (generated using Synthea), which statistically mirrors the real population in terms of demographics, disease burden, and healthcare interaction. This data is free of PII and PHI.
- [simstudy](https://cran.r-project.org/package=simstudy): an R package used to simulate datasets. The user specifies a set of relationships between covariates and type of study, and the tool generates synthetic data.
- [synthpop](https://cran.r-project.org/web/packages/synthpop/index.html): an R package that takes patient data with senstive values, and replaces those values with synthetic values, while minimally distorting the statistical summaries of the data set.

### Benefits of synthetic data

Using synthetic data has a variety of potential benefits.

- Reducing privacy risks: synthetic data are intended to have little or no information about individuals, and therefore reduce privacy risks associated with using clinical data. This also means they can more easily be shared across institutions and without needing to be granted access.
- Lower cost to generate: synthetic data typically cost less to generate than the cost to collect real patient data, and once an AI system has been trained to generate synthetic data then it can keep producing more. This is particularly useful for training AI systems, which need large sample sizes.

### Concerns about synthetic data

Although there is a lot of potential for synthetic data to revolutionize fields that use clinical data, there are also quite a few practical and ethical concerns.

- Accuracy and reliability: it is challenging to assess how well synthetic data actually mimic real data. Therefore, it is often unknown exactly how reliable statistical tests are based on synthetic data, as opposed to real patient data.
- Privacy risks: if synthetic data is built from real patient data, there are still threats of identifying patients in some cases, or small subgroups of patients in other cases.
- Bias: using AI systems in many applications has been shown to lead to algorithmic bias. It is possible for synthetic data to reinforce biases from original patient data or to create new biases.
- Lack of regulations: because synthetic data is often not considered to be personally identifiable information (PII) or protected health information (PHI), it is not regulated by the same guidelines. It is unclear what guidelines should regulate synthetic data, if any.
- Model collapse: when generative models are repeatedly trained on synthetic data rather than real data, the generated data may progressively diverge from true real-world patterns, reducing their utility (https://www.nytimes.com/interactive/2024/08/26/upshot/ai-synthetic-data.html).

Overall, while synthetic data has promise for several areas of clinical research, it needs to be carefully validated and used responsibly.

## Summary
Loading