Section 5: Experimental Design

Question 1

Experimental Control

Answer 1

(Functioanl Relations; Analysis; Control)

• When a predictable change in behavior [dependent variable-DV] can be reliably produced by the systematic manipulation of some aspect of the individual’s environment [independent variable-IV]
• The analysis dimension of the 7 dimensions of ABA

Question 2

4 Important Elements of Behavior

Answer 2

DICE

• Behavior is DETERMINED
  
  • the occurrence of any event is determined by the functional relations it holds to other events
  • behavior is a natural phenonmenon and subject to the same natural lawas as other natural phenomena
• Behavior is INDIVIDUAL
  
  • ​behavior is defined as a person’s interaction with the environment
  • Groups of people do not behave
  • Experimental strategy of ABA is based on single subject methods of analysis
• ​Behavior is CONTINUOUS
  
  • behavior changes over time
  • behavior requires continuous measurement over time
• Behavior variability is EXTRINSIC to the organism
  
  • variability (change in behavior) is the result of environemnt:
    
    • the IV under investigation
    • some uncotrolled aspect of the experiment (e.g. the behavior of another child in the classroom)
    • uncontrolled factor outside of experiment (e.g. weather changes)

Question 3

When we see variability in our data, what should we do?

Answer 3

• we should attempt to experimentally manipulate factors suspected of causing the variuability in the data to look for causal factors
• seek treatment variables robust enough to overcome variability

Question 4

6 Components of Experiments in ABA

Answer 4

MS. BEST

1. a measurement system and ongoing analysis of data
2. at least one subject
3. at least one behavior (DV)
4. an experimental design
5. at least one setting
6. at least one treatment (IV)

Question 5

Experimental Question

Answer 5

• a brief but specific statment of what the researcher wants to learn from conducting the experiment
• ex. what are the effects of the IV on the DV for what population and in what setting?
• ex. the purpose of the study was to see the effects of the IV on the DV.

Question 6

At Least One Subject (Single-Subject Designs)

Answer 6

(Single-Case Designs; Within-Subject Designs; Intra-Subject Designs)

(one of the 6 components of ABA experiments)

• ABA uses single-subject designs but thats does not mean there’s only 1 subject in the research
  
  • it’s called single subject because the subject as as their own control
    
    • ​repeated measures of the subject’s behavior during each phase of the study provide the basis for comparing experimental variables as they are presented or withdrawn in subsequent conditions (ie the presence and absence of the IV)
    • the individual is exposed to each condition several times over the course of a study
• studies usually involve more than 1 subject– usually 4 to 8
• Each subject’s data is graphed seperately
• ABA does not use group comparison designs (traditionally used in psychology) because group designs mask individual progress

Question 7

At Least One Behavior

Answer 7

(Dependent Variable)

(one of the 6 componenets of ABA experiments)

• The behavior is the dependent variable.
• DEPENDENT VARIABLE is the variable in the experiment measured to determine if it changes as a result of manipulations of the independent variable. In ABA, the DV must be a socially signficant behavior.
• some studies measure more than 1 DV. Reasons:
  
  • provide data patterns that can serve as controls for evaluating and replicating the effects of an IV
  • Assess if any COLLATERAL EFFECTS occured (a phenomenon in which the IV effects behaviors other than the targeted behavior)
  • determine whether changes in the behavior of a person other than the subject occur during the course of an experiment and if such changes can explain changes in the subject’s behavior
    
    • Ex. you are implementing an intervention for your client and the behavior of his brother changes as well
      ​

Question 8

Collateral Effect

Answer 8

A phenomenon in which the IV effects behaviors other than the targeted behavior

Question 9

At Least One Setting

Answer 9

(one of the 6 components of ABA experiments)

• Control 2 sets of enviornmental variables to demonstrate experimental control:
  
  • IV (present, withdraw, or vary its value)
  • Extraneous Variables (prevent unplanned environmental variation)
• In labs, we can control enviornments better
• When unplanned variations take place, you can wait them out or incorporate them into the design
  
  • repeated measures of behavior tell us whether unplanned environmental changes are of concern

Question 10

At Least One Treatment

Answer 10

(Independent Variable; Intervention; Experimental Variable)

(one of the 6 components of ABA experiments)

• The IV is the variable that is systematically manipulated by the researcher in an experiment to see whether changes in the independent variable produce reliable changes in the dependent variable.
• …. a particular aspect of the environment that the experimenter manipulates to find out whether it affects the subject’s behavior

Question 11

A Measurement System and Ongoing Analysis of Data

Answer 11

(one of the 6 components of ABA experiments)

• observation and recording procedures must be conducted in a standardized manner
• standardization involves every aspect of the measurement system (e.g. from the behavior definition to scheduling of observations)
• behaviorists must detect changes in level, trend, and variability

Question 12

Experimental Design

Answer 12

• the particular arrangement of conditions in a study so that meaningful comparisons of the effects of the presence, absence, or different values of the IV can be made
• Important Rules of Experimental design
  
  • change only one variable at a time
    
    • if you are examining a treatment package (behavioral package), ensure the entire package is presented or whithin drawn at the same time
  • select and combine designs that best fit the research question

Question 13

Two Types of Experimental Designs

Answer 13

Para = Pyramid of Values

Nonparametric Analysis- the IV is either present or absent during the study (ex. medication is either given or not given in a drug study)

Parametric Analysis - The value of the IV is manipulated. This seeks to discover the differential effects of a range of values. (ex. various doses of medication are given throughout the study)

Question 14

Treatment Package

Answer 14

• (Behavioral Package)*
• when multiple IVs are bundled together into one program (e.g. a token economy with praise and time out procedures)

Question 15

Component Analysis

Answer 15

• a process that looks at the effect of each part of a treatment package
• used to determine the effective components of a treatment package

Question 16

Steady State Responding

Answer 16

(Stable State Responding)

• a pattern of responding that exhibits very little variation in its measured dimensional quantities over a period of time
• Provides the basis for baseline logic

Question 17

Baseline Logic

Answer 17

• refers to the experimental reasoning inherent in single subject experimental designs
• 3 elements (PAVR)
  
  • Prediction
  • Affirmation of the Consequent
  • Verification
  • Replication
• each of these elements depends on an overall experimental approach called steady state strategy

Question 18

Steady State Strategy

Answer 18

repeated exposure of a given subject to a given condition while trying to eliminate extraneous influcences on behavior and obtaining a stable pattern of responding before introducing the next condition

Question 19

Function of Baseline Data

Answer 19

It serves as a control condition but does NOT imply the absence of intervention–it can be the absence of a specific IV.

Question 20

Benefits of Baseline Data

Answer 20

• to use the subject’s performance in the absence of the IV as an objective basis for detecting change
• to obstain descriptions of ABC correlations for the planning of an effective treatment
• to guide us in setting the initial criteria for reinforcement
• to see if the bheavior targeted for change really warrants intervention

Question 21

4 Patterns of Baseline Data

Answer 21

DAVS

1. Descending Baseline - don’t implement the IV unless its a behavior (functional skill that is being lost) that you need to increase
2. Ascending Baseline - don’t implement the IV unless it’s a behavior you need to decrease (challenging behavior)
3. Variable Baseline - don’t implement the IV until you’ve controlled all of the environmental variables that may be causing the variability
4. Stable Baseline - implement the IV when all of the values of the DV small in a fall range of values

Question 22

Prediction

Answer 22

(the first part of baseline logic)

• the anticipated outcome of a presently unknown measurement
• data should be collected until stability is clear–the more data the better the predictive power
• main question to ask yourself: are data stable enough to serve as the basis for experiental comparison?

Question 23

Affirmation of the Consequent

Answer 23

PREDICT THEN AFFIRM YOUR PREDICTION

(the 2nd part of baseline logic)

• Inductive Logic
  
  • If the IV were not applied, the behavior (as indicated by baseline data) would not change
  • the experimenter predicts the IV will change the behavior
  • if the IV is controlling the DV, the data path in the presence of the IV will show that the DV has changed
  • when the IV is present, data show DV has changed
  • thus the IV is controlling the DV

Question 24

Verification

Answer 24

VERIFY THE EFFECTS OF THE IV

(the third part of baseline logic)

• Verification of a previously predicted level of baseline responding by termination or withdrawal of the treatment variable

Question 25

Replication

Answer 25

(the fourth part of baseline logic) * replication is the essence of **believability** * shows **reliability** of behavior change * replication is accomplished by reintroducing the IV

Question 26

Ethical Issues regarding experimental designs and research

Answer 26

* CHECK THE CODE

Question 27

5 Main Experimental Designs

Answer 27

_MC RAW_ 1. Multiple Baseline 2. Changing Criterion 3. Reversal 4. Alternating Treatments 5. Withdrawal

Question 28

Multiple Baseline Design

Answer 28

* most widely used design * highly flexible * staggered implentation of the intervention in a step wise fashion across behaviors, settings, and subjects * do NOT have to withdraw a treatment variable in the design * ETHICS WARNING- when it is unethical or impractical to reverse conditions or when the behavior is irreversible, use this design instead of a reversal design * 3 Main Types * Multiple Baseline Across Behaviors * Multiple Baseline Across Settings * Multiple Baseline Across Subjects * Two Weaker Variations- used when extended baseline measurement is unnecessary, impractical, too costly, or unabailable * Multiple Probe Design * Delayed Multiple Baseline Design

Question 29

Prediction, Verification, and Replication in the Multiple Baseline Design

Answer 29

* **a functional relation requires a change in behavior with the onset of the intervention** * apply the IV to behavior 1 when you can confidiently **predict** that the behavior would remain the same in constant conditions * if behaviors 2 and 3 remain unchanged after the application of the IV to behavior 1, this **verifies** the prediction * if the IV changes behavior 2 like it did behavior 1, the effect of the IV has been **replicated** * the more replications, the more convincing the demonstration * most mutliple baseline designs have 3 to 5 tiers

Question 30

Multiple Baseline Across Behaviors

Answer 30

* two or more different behaviors of the SAME SUBJECT * each subject serves as their own control * after steady state baseline responding, the IV is applied to the first behavior while other behaviors are kept in baseline * when steady state responding is reached for the first behavior, then the IV is applied to the next behavior

Question 31

Multiple Baseline Across Settings

Answer 31

* **a single behavior** is targeted in two or more settings or conditions * after steady state baseline responding, the IV is applied to the first setting while the other settings are kept in baseline * when steady state responding is reached for the first setting, then the IV is applied to the next setting

Question 32

Multiple Baseline Across Subjects

Answer 32

* one target behavior for two or more subjects in the **same setting** * after steady state baseline responding is reached, the IV is applied to the first subject, while other subjects are kept in baseline * when steady state responding is reached for the first subject, then the IV is applied to the next subject * **most widely used multiple baseline design**

Question 33

Multiple Probe Design

Answer 33

* analyzes relation between the IV and acquistion of skill sequences * instead of stimultaneious baselines, probes provide the basis for determining if behavior change has occured prior to intervention

Question 34

Delayed Multiple Baseline Design

Answer 34

* initial baseline and intervention begin and subsequent baselines are added in a delayed or staggered fashion * effective when * reversal design is not possible * limited resources preclude a full scale design * when a new behavior, subject, or setting becomes available * limitations- shorter baselines do not show interdependence of DVs

Question 35

Guidelines for Multiple Baseline Design

Answer 35

* **Select independent, yet functionally similar baselines** * behaviors are functionally independent of one another * behaviors share enough similarity that they will change with the application of the same IV * behaviors should be of different response classes * **select concurrent and plausibly related multiple baselines** * behaviors must be measured concurrently * all relevant variables that influence one behavir must have the opportunity to influence other behaviors * **Do not apply the IV to the next behavior too soon** * **Significantly vary the lengths of multiple baselines** * the more the baselines differ in length, the stronger the design * **Intervene on the most stable baseline first**

Question 36

Advantages and Disadvantages of Multiple Baseline Design

Answer 36

Advantages * successful intervention does not have to be removed * evaluates generalization * easy to implement Disadvantages * functional relation is not directly shown in this design (as compared to reversal) * effectiveness of the IV is demonstrated but not information regarding the function of the target behavior * IV may be delayed for certain behaviors, settings, or subjects * takes resources to implement properly

Question 37

Changing Criterion Design

Answer 37

* Experimental design in which an initial baseline phase is followed by a series of treatment phases consisting of successive and gradually changing criteria for reinforcement or punishment * only one behavior in this design * behavior must be in the subjects repertoire already * evaluates treatment that is applied in a graduated or step wise fashion -- can be changes in rate, accuracy, duration, or latency of a single target behavior * technically, it is a variation of the multiple baseline design

Question 38

Prediction, Verification, and Replication in the Changing Criterion Design

Answer 38

* the criterion lines should have a large separation to show a functional relationship * experimental control is evidenced by the extent that the level of responding changes to conform to each new criterion * if data points do not fall around the criterion lines, that shows us that there is very little experimental control * the greater the vertical distance between the criterion lines, the more experimental control

Question 39

Guidelines for Changing Criterion Designs

Answer 39

1. Length of Phases * each phase must be long enough to achieve stable responding * target behaviors that are slower to change require longer phases * validity of the design is increased when you **vary the length of each phase** 2. Magnitude of Criterion Changes * the **size of the changes between each criterion should vary** to prove strong functional relations * changes in size must be large enough to be detectable but not so large as to be unachievable * changes in size can be smaller if you are dealing with stable data 3. Number of Criterion Changes * the more criterion changes the better proof of experimental control

Question 40

Advantages and Disadvantages of Changing Criterion Designs

Answer 40

* Advantages * does not require reversal of improved behavior * enables an experimental analysis within the context of a gradually improving behavior -- you don't have to wait for a stable baseline * Disadvantages * the target behavior must already be in the person's repertoire * not a comparison design * not appropriate for analyzing the effects of a shaping program-- WHY? * shaping is a behavior change strategy--not an experimental design * shaping is used to teach novel behaviors

Question 41

Reversal Design

Answer 41

*(A-B-A; B-A-B Design)* * any experimental design in which the researcher reverses responding to a level obtained in a previous condition * **the most powerful within subject design for demonstrating function** * each reversal strengthens experimental control * eivdence of a functional relation is strengthened with each reversal switch from one condition to the other with a corresponding change in trend and level * for a reversal to occur, the behavior must approximate the initial baseline levels * Requires at least 3 consecutive phases * Initial Baseline (A) * Intervention (B) * Return to Baseline (A) * 5 Reversal Designs * Repeated Reversals * BAB * ETHICS WARNING- if your client is displaying sever and dangerous behaviors, then do NOT spend time just taking baseline data. it is your ethic responsiblity to start treatment immediately. So, use the BAB design * Multiple Treatment Design * NCR Reversal Technique * DRO/DRA/DRI reversal technique

Question 42

Prediction, Verification, and Replication in the Reversal Design

Answer 42

* the IV is responsible for behavior change if repetition of baseline and treatment phases approximate the original phases

Question 43

5 Variations of the Reversal Design

Answer 43

1. Repeated Reversal * simple extension of A-B-A-B * the more reversals, the stronger your evidence of control, but redundancy may be a concern 2. BAB Reversal * weaker than an A-B-A design because it does not enable assessment of th effects of the IV during baseline * also subject to **SEQUENCE EFFECTS** because baseline may have been influenced by starting with the IV * best used when dealing with challenging behavior that needs immediate intervention or when an IV is already in place 3. Multiple Treatment Design * compares two or more IVs to baseline and/or to one another * ABACABAC * ABCDACAD * disadvantage- sequence effects 4. NCR Reversal Technique * uses NCR as a control condition instead of a baseline condition in which no reinforcement is provided * allows you to examine contingent reinforcement 5. DRO/DRI/DRA Reversal Technique * uses DRO, DRA, or DRI as a control condition instead of a baseline condition in which no reinforcement is provided * allows you to examine contingent reinforcement

Question 44

Sequence Effects

Answer 44

*(Carryover Effects; Alteration Effects)* * effects on a subject's behavior in a given condition that are the result of the subject's experience with a prior condition * ex. if you are using an ABCBC design, you cannot anlayze the effects of C on its own because it is always preceded by B. If you want to analyze C, you need to add ACAC

Question 45

Advantages and Disadvantages of Reversal Design

Answer 45

Advantages * clear demonstration of the existence or absence of a functional relation between IV and DV * enables us to count the amount of behavior change * return to baseline tells us we need to program for maintenance Disadvantages * Irreversibility * Sequence Effects * ETHICS WARNING- ethical issues as well as social and educational issues arise when you remove an effective IV

Question 46

Irreversibility

Answer 46

* the level of behavior observed in an earlier phase cannot be reproduced even though experimental conditions are the same as they were during the earlier phase * when irreversibility is a problem, use DRO/DRI/DRA conditions as control techniques or multple baseline designs * ex. learning to ride a bike is irreversible

Question 47

Alternating Treatments Design

Answer 47

*(SCAMM:* ***S**imultaneous Treatments Design; **C**oncurrent Schedules Design; **A**lternating Treatments Design; **M**utli Element Baseline Design; **M**utli Element Design; **M**ultiple Schedules Design)* * an experimental design in which two or more conditions are presented in rapidly alternating succession independent of th elevel of responding and the differential effects on the target behavior are noted * compares two or more iVS to one another to see which IV would be best to utilize with a client * based on **stimulus discrimination** - each IV has an obvious S^D which is in effect * For each IV, data are plotted separately on the same graph * IVs may be * alternated across daily sessions * given in sessions occuring the same day * implemented during each portion of the same session

Question 48

Prediction, Verification, and Replication in the Alternating Treatments Design

Answer 48

On Graphs * visual inspection of the differences between or among the data paths produced by each treatment * functional relation is shown when * one data path is consistently higher than the other * no overlapping data paths * the degree of differential effects produced by two different treatments is determined by the **vertical distance between the respective data paths** * each successive data point in treatment plays all three roles- prediction, replication, verification

Question 49

3 Variations of Alternating Treatments Design

Answer 49

1. Alternating Treatment Designs Single Phase Without Baseline * does not require an initial baseline 2. Alternating Treatment Designs With Baseline * whenever possible, you should use a baseline 3. Alternating Treatment Designs With Baseline and Final Best Treatment Phase * most widely used

Question 50

Advantages and Disadvantages of Alternating Treatments Design

Answer 50

Advantages * three problems avoided: 1. Irreversibility 2. Sequence Effects 3. Unstable Data * does not require treatment withdrawal * speedy comparison * can be used to assess generalization effects * intervention can begin immediately without baselin data Disadvantages * **multiple treatment interference**- two or more IVs in effect at the same time * unnatural nature of rapdily alternating treatments * limited capacity--maximum comparison of 4 conditions * selection of treatments- the effects should be significantly different from one another

Question 51

Withdrawal Design

Answer 51

* Withdrawal design is a synonym for ABAB or experiments when an effective treatment is sequentially or partially withdrawn to promote the maintenace of behavior changes * some researchers use "withdrawal" to describe experiments based on ABAB analysis and reserve the term "reversal design" for studies in which the behavioral focus of the treatment variable is reversed (or switched to another behavior) as in the DRO/DRI/DRA reversal techniques * those literally "reverse" treatments but doesn't actually completely "withdraw" all treatment * it's most common in the literature to see "reversal design" encompassing "withdrawal designs" since both attempt to demonstrate behavioral reversibility

Question 52

How to Identify Practical and Ethical Considerations in Single Case Experimental Design to Demonstrate Treatment Effectiveness

Answer 52

1. Baseline Trends * increasing or decreasing trends in your data during baseline do NOT allow you to clearly demonstrate that your IV causes the change in behavior * Address by: * continue observations for a longer period of time * try to reverse the trend (e.g. by using a DRO) * select designs that do not require a stable baseline * use statistical techniques that take initial trends into account 2. Excessive Variability in Data * Address by: * block consectuvie data points and plot blocked averages rather than day to day performance * search for the causes of variability 3. Duration of the Phases * the duration of each phase in your design can involve problems related to rends and variability in the data * no rigid rules for how many data points you need in each phase, but you want steady responding * use objective criteria for deciding when to shift phases to reduce subjectvity 4. Other Concerns * the range of questions about intervention effects that can be addressed with these designs--they are best to evaluate treatment packages * the generality of the research results * is the finding generalized beyond the subject in the design? * to assess generality, use replication of your IV across subjects, etc.

Question 53

Why ABA has problems with traditional psychology research?

Answer 53

* group data is not representative of individual performance * group data masks variability * hides variability that occurs within and between subjects * statistical control is not a substitute for experimental control * to control effects of any variable you must either hold it constant or manipulate it as an IV * absence of intrasubject replication

Question 54

Two Types of Validity in Experimental Designs

Answer 54

Internal Validity External Validity

Question 55

Internal Validity

Answer 55

_IV (internal validity) = IV (independent variable)_ * the extent to which an experiment shows convincingly that changes in behavior are a function of the IV and not the result of uncontrolled or unknown variables * an internally valid study involves only one IV at a time. Multiple IVs are not confounded (presented at the same time) * High Internal Validity = Designs showing strong experimental control

Question 56

4 Confounding Threats to Internal Validity

Answer 56

_MISS_ 1. **Measurement Confounds**- refers to the number and intricacy of the behaviors you are targeting. If you are targeting too many complicated behaviors, your internal validity may be affected 1. **Observer Drift**- when observers unknownly alter the way they apply a measurement system 2. **Reactivity**- this can refer to the bheavior of clients changing when they are being observed (to reduce, run a longer baseline) 3. **Observer bias/expectations**- the observer's expectations that changes follow a certain direction (to reduce, have naive observers) 2. **IV Confounds**- when IVs are complicated and implemented together in a treatment package * ex. giving money as a reinforcer is also giving the person attention so you can't be sure if the maintaining reinforcer is the actual money or the attention * to reduce- use a placebo control or double blind control procedure in which the subject is not aware if the IV is present or not 3. **Subject Confounds** * **Maturation** - changes in subjects over the course of the study * to reduce- repeat measurements 4. **Setting Confounds** * studies in natural settings will have more confounds than in labs * hold all possible aspects of the study constant until repeated measurements again reveal stable responding * **Bootleg Reinforcement** (secretive reinforcement that is not part of your behavior plan) may also occur in the natural enviornment

Question 57

Confounding Variables

Answer 57

*(Extraneous variables; Unrelated Variables)* * generally confounding and extraneous variables are AKAs that refer to variables that exert an uncontrolled influence on a study * Extraneous = an aspect of the ENVIRONMENT that must be held constant * ex. lighting, space, temperature in the room * Confounding = any uncontrolled factor known or suspected to exert influence on the dependent variable

Question 58

External Validity

Answer 58

_External Validity = Generalizable to the External World_ * the degree to which a study's results are generalizable to other subjects, settings, and/or behaviors * the degree to which a functional relation discovered in a study will hold under different conditions * replication establishes external validity * **Direct Replication** * exactly duplicate the study * **Intrasubject direct replication** - same subject is used * **Intersubject direct replication**- different subjects used * **Systematic Replication** * researcher purposefully varies one or more aspects of an earlier experiment * deomonstrates reliability and external validity by showing the same effect can occur under different conditions * ABA uses systematic replication

Question 59

Treatment Integrity

Answer 59

*(Procedural Fidelity; Fidelity of Implementation; Program Integrity)* * extent to which the IV is implmented or carried out as planned * low treatment integrity - very difficult to interpret experimental results * **treatment drift**- when application of the IV in later phases differs from the original application

Question 60

How to Assess and Ensure a High Level of Treatment Integrity

Answer 60

* Ensure high level of treatment integrity by: * use precise operational defintion of treatment procedures * simplify, standardize, and automate treatments--more likely to be consistently delivered and also socially validated * training and practice for individuals who will conduct the experimental sessions (e.g. detailed scripts and instructions) * Assessing: * collect treatment integrity data to measure how the actual implementation of the conditions matches the written methods * observation and calibration give the researcher the ongoing ability to use retraining and practice to ensure high treatment integrity * reduce, eliminate, or identify the influence of potential confounding variables

Question 61

2 Types of Errors in Evaluating ABA Research

Answer 61

* Type I Error *(False Positive)* * assuming the IV affected the DV when it did NOT actually do so * Statistical Analysis tends to lead to more Type 1 errors * Type 2 Error *(False Negative)* * assuming the IV did NOT affect the DV when it actually DID * Visual analysis in ABA tends to lead to more Type II errors