Control groups

(% class="lead" %)

The control group mechanism helps evaluate the impact of communications on customer behavior within a segment by comparing metrics (purchase conversion, average receipt value, average revenue per customer) between customers who received the communication and those who did not.

3

4

This mechanism allows you to isolate a small subset—known as the control group—from the target communication segment. Typically, this represents 5–10% of the segment size. This **control group** does not receive the communication, yet the platform calculates the same metrics for them as it does for the main group that received the communication.

5

6

During post-analysis, the platform estimates what the revenue in the main group would have been if its metrics matched those of the control group (i.e., if the main segment had not received the communication).

7

8

== Creating a control group ==

9

10

Control groups can be used in all types of [[mass messaging>>doc:Main.Smart_Communications.SMC_Use.Mass_messaging.WebHome]] (email, SMS, push) and in [[campaigns>>doc:Main.Smart_Communications.SMC_Use.Campaigns.WebHome]].

11

12

=== In mass messaging ===

13

14

When [[creating a new mailing>>doc:Main.Smart_Communications.SMC_Use.Mass_messaging.WebHome]], after adding customers to the segment, the **Messaging volume** block appears. To create a control group, use the **Control group size** slider in the **Messaging volume** block.

15

16

|(% style="border-color:white; text-align:center" %){{lightbox image="Control_Group.png" width="1200"/}}

17

18

You can define the control group size in two ways:

19

20

* **By percentage** — ranging from 0% to 50% of the target segment (default is 0%). The number of customers included in the control group is displayed.

21

* **By target audience**. When selecting this option, a list of created [[target audiences>>doc:Main.Smart_Communications.SMC_Use.Customers.Target_audiences.WebHome||anchor="H41443E43143043243844244C43243044343443844243E44043844EA0"]] appears for selection as the control group.

=== In campaigns ===

|(% style="border-color:white; width:840px" %)(((

26

When creating a campaign, click the control group icon to set up a control group. In the pop-up window, specify the **Control group size**:

27

28

* **as a percentage**, using a slider ranging from 0% to 50% of the target segment (default is 0%);

29

* by selecting a [[target audience>>doc:Main.Smart_Communications.SMC_Use.Customers.Target_audiences.WebHome]] from the dropdown list as the control group.

30

31

(% class="box infomessage" %)

32

(((

33

A customer's assignment to the control group is determined at the moment the [[trigger>>doc:Main.Smart_Communications.SMC_Use.Campaigns.Triggers.WebHome]] fires. Therefore, if the campaign includes branching logic (e.g., an [[A/B test>>doc:Main.Smart_Communications.SMC_Use.Campaigns.Actions.WebHome||anchor="HA2FBtest"]]), customers in the control group may be distributed unevenly across branches.

34

35

Similarly, if the campaign uses the [[Filter customers>>doc:Main.Smart_Communications.SMC_Use.Campaigns.Actions.WebHome||anchor="HFiltercustomers"]] action, the proportion of control group customers in the final communication may be lower than planned.

)))

)))|(% style="border-color:white; width:598px" %){{lightbox image="Control_G_Camain.png" width="300"/}}

40

41

=== Stratification parameters {{id name="Stratification"/}} ===

42

43

When configuring control groups within [[mass messaging>>doc:Main.Smart_Communications.SMC_Use.Mass_messaging.WebHome]] and [[campaigns>>doc:Main.Smart_Communications.SMC_Use.Campaigns.WebHome]], you can specify **stratification parameters**. Stratification is a two-step process where the [[target audience>>doc:Main.Smart_Communications.SMC_Use.Customers.Target_audiences.WebHome]] is first divided into subgroups (strata) based on key characteristics or **stratification parameters** (e.g., age, gender, registration date, days since last receipt). Then, within each subgroup, customers are randomly assigned to the main and control groups. Stratification ensures that the control group is a representative replica of the main group across key parameters, enabling accurate measurement of communication impact.

44

45

Stratification parameters include attributes from the **Customer** filter group in the [[Segment builder>>doc:Main.Smart_Communications.SMC_Use.Customers.User_segments.Segment_builder.WebHome]].

46

47

|(% style="border-color:#ffffff; text-align:center" %)[[image:attach:RFM.png]]

48

49

For example, stratification can be performed using the following parameters:

50

51

* **Current R-segment** — number of days since the customer’s last purchase;

52

* **Current F-segment** — number of purchases made during the segmentation period;

53

* **Current M-segment** — total purchase amount during the segmentation period.

54

55

The system divides the [[target audience>>doc:Main.Smart_Communications.SMC_Use.Customers.Target_audiences.WebHome]] into the specified subgroups and randomly allocates customers within each subgroup to the main and control groups according to the percentage defined in the control group settings.

56

57

== {{id name="report"/}}Control group reports ==

58

59

The control group report is available on the page containing all other [[mailing reports>>doc:Main.Smart_Communications.SMC_Use.Mass_messaging.Mailing_report.WebHome]] or [[campaign reports>>doc:Main.Smart_Communications.SMC_Use.Campaigns.Report.WebHome]].

60

61

|(% style="border-color:white; text-align:center" %){{lightbox image="screen1.png"/}}

62

63

The table displays the following information for each group:

64

65

* **Number of recipients** — how many messages were sent to this group;

66

* **Messages delivered** — how many messages were successfully delivered to this group;

67

* **Number of customers who made a purchase**;

68

* **Purchase conversion rate** — the percentage of customers who made a purchase influenced by the mailing;

69

* **Number of receipts in the group**;

70

* **Average number of receipts per customer**;

71

* **Total purchase amount**;

72

* **Average revenue per customer**;

73

* **Revenue forecast assuming control group conversion and average receipt value** — estimated revenue if the mailing had not been sent:

74

** Calculated only for the main group.

75

** Equals (Messages delivered) × (Purchase conversion rate in control group) × (Average revenue per customer in control group).

76

77

For example, the report shows:

78

79

23,056 people received the mailing. Without the mailing, the purchase conversion would have been 1.09%, as observed in the control group. With this conversion rate, 23,056 × 0.0109 = 251 customers would have made a purchase. The average revenue per customer in the control group is RUB 9,435. Thus, without the mailing, total purchases would amount to 251 × 9,435 = RUB 2,368,185.

80

81

* **Additional revenue** — the incremental benefit generated by the mailing:

82

** Equals (Total purchase amount in main group) – (Revenue forecast assuming control group conversion and average receipt value).

83

84

For example, total purchases in the main group amount to RUB 3,453,671, while the forecasted revenue is RUB 2,379,208. The difference is RUB 1,074,463.

85

86

(% class="box infomessage" %)

87

(((

88

The report displays receipts attributed to the mailing. For more details on attribution, see [[Receipt attribution>>doc:Main.Smart_Communications.SMC_Use.Receipt_attribution.WebHome]].

89

)))

90

91

== Control groups in the Segment builder ==

92

93

In the [[Segment builder>>doc:Main.Smart_Communications.SMC_Use.Customers.User_segments.Segment_builder.WebHome]], you can use filters related to control groups (CG):

94

95

* Participated in CG of [[email mass messaging>>doc:Main.Smart_Communications.SMC_Use.Mass_messaging.Email_mass_messaging.WebHome]];

96

* Participated in CG of [[SMS mass messaging>>doc:Main.Smart_Communications.SMC_Use.Mass_messaging.SMS_mass_messaging.WebHome]];

97

* Participated in CG of [[push mass messaging>>doc:Main.Smart_Communications.SMC_Use.Mass_messaging.Push_mass_messaging.WebHome]];

98

* Participated in CG of [[campaign>>doc:Main.Smart_Communications.SMC_Use.Campaigns.WebHome]].

99

100

For mailings/campaigns with a control group, the filters **Participated in email mailing**, **Participated in SMS mailing**, **Participated in push mailing**, and **Participated in campaign** exclude control group members from the selection.

101

102

To select the entire mailing/campaign segment (CG + main group), apply two filters combined with the logical operator **OR**. Example:

103

104

|(% style="border-color:white; text-align:center" %){{lightbox image="Segment.png" width="1200"/}}

(% class="box" %)

(((