New kb #238
New kb #238
Conversation
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
docs/glossary.mdx (1)
273-273: Minor formatting and hyphenation corrections in the Multivariate experiment entry.Lines 273–282 contain a few formatting improvements:
- Line 277: Remove the colon after "with" (you should not use a colon after a preposition introducing a series).
- Line 280: Hyphenate "call-to-action" (should be "call‑to‑action buttons").
These are minor but worth correcting for consistency with standard MDX/documentation conventions.
Also applies to: 277-277, 280-280
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
docs/glossary.mdx(1 hunks)
🧰 Additional context used
🪛 LanguageTool
docs/glossary.mdx
[grammar] ~15-~15: If ‘type’ is a classification term, ‘A’ is not necessary. Use “type of”. (The phrases ‘kind of’ and ‘sort of’ are informal if they mean ‘to some extent’.)
Context: ... experiment Definition: A special type of A/B experiment where users are randomly s...
(KIND_OF_A)
[style] ~17-~17: Would you like to use the Oxford spelling “randomization”? The spelling ‘randomisation’ is also correct.
Context: ...M), tracking bugs or unexpected bias in randomisation before you start testing real changes. ...
(OXFORD_SPELLING_Z_NOT_S)
[grammar] ~27-~27: If ‘type’ is a classification term, ‘A’ is not necessary. Use “type of”. (The phrases ‘kind of’ and ‘sort of’ are informal if they mean ‘to some extent’.)
Context: ... experiment Definition: A special type of A/B experiment that compares a control to...
(KIND_OF_A)
[style] ~34-~34: Would you like to use the Oxford spelling “generalizability”? The spelling ‘generalisability’ is also correct.
Context: ...eting can reduce sample size and affect generalisability. Example: Running an experiment onl...
(OXFORD_SPELLING_Z_NOT_S)
[uncategorized] ~42-~42: Possible missing comma found.
Context: ...scribing outcomes that have exactly two states such as success / failure or convert / ...
(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~70-~70: Loose punctuation mark.
Context: ... more informative than a p-value alone. :contentReference[oaicite:5]{index=5} *...
(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~90-~90: Loose punctuation mark.
Context: ...ods are designed to handle this safely. :contentReference[oaicite:6]{index=6} *...
(UNLIKELY_OPENING_PUNCTUATION)
[grammar] ~104-~104: The verb ‘covariate’ does not usually follow articles like ‘a’. Check that ‘covariate’ is spelled correctly; using ‘covariate’ as a noun may be non-standard.
Context: ...using correlated pre-experiment data as a covariate. Why it matters: CUPED can signif...
(A_INFINITIVE)
[uncategorized] ~105-~105: Use a comma before ‘so’ if it connects two independent clauses (unless they are closely connected and short).
Context: ...ED can significantly improve sensitivity so experiments reach conclusions faster or...
(COMMA_COMPOUND_SENTENCE_2)
[uncategorized] ~105-~105: Loose punctuation mark.
Context: ...smaller effects using the same traffic. :contentReference[oaicite:7]{index=7} *...
(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~112-~112: The noun “decision-making” (= the process of deciding something) is spelled with a hyphen.
Context: ...g the test. --- ## D ### Data-driven decision making Definition: A decision process th...
(DECISION_MAKING)
[uncategorized] ~123-~123: Loose punctuation mark.
Context: ...g with the same experimentation budget. :contentReference[oaicite:8]{index=8} *...
(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~128-~128: Loose punctuation mark.
Context: ...ady clear, preserving traffic and time. :contentReference[oaicite:9]{index=9} *...
(UNLIKELY_OPENING_PUNCTUATION)
[style] ~133-~133: To elevate your writing, try using a synonym here.
Context: ...actions can distort results and make it hard to attribute observed effects to a sing...
(HARD_TO)
[uncategorized] ~138-~138: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...ul changes will show up as significant; low power leads to many missed opportunities. *...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
[style] ~144-~144: Would you like to use the Oxford spelling “generalizes”? The spelling ‘generalises’ is also correct.
Context: ... country site to verify that the effect generalises. ### Exploratory vs confirmatory exper...
(OXFORD_SPELLING_Z_NOT_S)
[style] ~147-~147: Try moving the adverb to make the sentence clearer.
Context: ...irections; confirmatory experiments are designed to rigorously test a specific hypothesis. Why it matters: Mixing the two m...
(SPLIT_INFINITIVE)
[uncategorized] ~157-~157: Loose punctuation mark.
Context: ... rate of wrong “winners” under control. :contentReference[oaicite:10]{index=10} ...
(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~182-~182: Possible missing comma found.
Context: ... horizon methods are conceptually simple but are not robust to unplanned peeking or ...
(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~182-~182: Loose punctuation mark.
Context: ...to unplanned peeking or early stopping. :contentReference[oaicite:11]{index=11} ...
(UNLIKELY_OPENING_PUNCTUATION)
[style] ~183-~183: ‘making a decision’ might be wordy. Consider a shorter alternative.
Context: ...an experiment for exactly two weeks and making a decision only after both weeks have completed. ...
(EN_WORDINESS_PREMIUM_MAKING_A_DECISION)
[uncategorized] ~187-~187: Loose punctuation mark.
Context: ...more complex design and interpretation. :contentReference[oaicite:12]{index=12} ...
(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~192-~192: Loose punctuation mark.
Context: ...be reallocated to more promising ideas. :contentReference[oaicite:13]{index=13} ...
(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~197-~197: Possible missing comma found.
Context: ...her to redesign the experiment, improve metrics or simply move on to other ideas. **E...
(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~205-~205: Specify a number, remove phrase, use “a few”, or use “some”
Context: ...sequential approach where you predefine a small number of checkpoints (interim looks) at which yo...
(SMALL_NUMBER_OF)
[uncategorized] ~206-~206: Loose punctuation mark.
Context: ...here a few well-timed looks are enough. :contentReference[oaicite:14]{index=14} ...
(UNLIKELY_OPENING_PUNCTUATION)
[style] ~219-~219: Would you like to use the Oxford spelling “randomized”? The spelling ‘randomised’ is also correct.
Context: ... anecdotes and observational data up to randomised controlled experiments and meta-analyse...
(OXFORD_SPELLING_Z_NOT_S)
[style] ~220-~220: Would you like to use the Oxford spelling “prioritize”? The spelling ‘prioritise’ is also correct.
Context: ...* The hierarchy of evidence helps teams prioritise decisions based on how reliable and cau...
(OXFORD_SPELLING_Z_NOT_S)
[style] ~234-~234: Would you like to use the Oxford spelling “prioritize”? The spelling ‘prioritise’ is also correct.
Context: ...Why it matters:** Impact estimates help prioritise which experiments to run and inform sam...
(OXFORD_SPELLING_Z_NOT_S)
[uncategorized] ~253-~253: Loose punctuation mark.
Context: ...e and you miss meaningful improvements. :contentReference[oaicite:15]{index=15} ...
(UNLIKELY_OPENING_PUNCTUATION)
[style] ~272-~272: Would you like to use the Oxford spelling “optimize”? The spelling ‘optimise’ is also correct.
Context: ...best. They are useful when you want to optimise several components together, such as he...
(OXFORD_SPELLING_Z_NOT_S)
[formatting] ~273-~273: If the ‘because’ clause is essential to the meaning, do not use a comma before the clause.
Context: ...ntly more traffic than a simple A/B test, because traffic must be spread across many vari...
(COMMA_BEFORE_BECAUSE)
[typographical] ~273-~273: Use a comma after an introductory phrase.
Context: ...tions and the analysis is more complex. For this reason Multivariate experiments are not suppor...
(COMMA_INTRODUCTORY_WORDS_PHRASES)
[style] ~277-~277: Would you like to use the Oxford spelling “optimize”? The spelling ‘optimise’ is also correct.
Context: ... ABsmartly. Example: You want to optimise a landing page with: - 2 different he...
(OXFORD_SPELLING_Z_NOT_S)
[typographical] ~277-~277: Do not use a colon (:) before a series that is introduced by a preposition (‘with’). Remove the colon or add a noun or a noun phrase after the preposition.
Context: ...* You want to optimise a landing page with: - 2 different headlines (H1, H2) - ...
(RP_COLON)
[grammar] ~280-~280: It appears that two hyphens are missing.
Context: ...ero images (I1, I2, I3) - 2 different call to action buttons (C1, C2) A multivariate expe...
(CALL_TO_ACTION_HYPHEN)
[grammar] ~304-~304: Did you mean “sample”?
Context: ...ble from the data, but it is subject to sampling variability. Example: Treatment s...
(IT_IS_JJ_TO_VBG)
[uncategorized] ~309-~309: Possible missing comma found.
Context: ... more powerful for directional questions but must be justified in advance to avoid b...
(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~315-~315: Would you like to use the Oxford spelling “utilization”? The spelling ‘utilisation’ is also correct.
Context: ... Example: Error rate, latency, CPU utilisation or cache hit rate. --- ## P ### Peek...
(OXFORD_SPELLING_Z_NOT_S)
[uncategorized] ~323-~323: Loose punctuation mark.
Context: ...make p-values and intervals unreliable. :contentReference[oaicite:16]{index=16} ...
(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~348-~348: Possible missing comma found.
Context: ...xperimentation is not a one-off activity but a core part of how the organisation bui...
(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~348-~348: Would you like to use the Oxford spelling “organization”? The spelling ‘organisation’ is also correct.
Context: ...off activity but a core part of how the organisation builds products. Example: Marty C...
(OXFORD_SPELLING_Z_NOT_S)
[uncategorized] ~353-~353: Loose punctuation mark.
Context: ... the probability that the null is true. :contentReference[oaicite:17]{index=17} ...
(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~373-~373: Possible missing comma found.
Context: ...oring average order value and page load time while the primary metric is conversion ...
(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~383-~383: Possible missing comma found.
Context: ...* Discounts shown only to the treatment group affecting reference prices for control ...
(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~387-~387: Loose punctuation mark.
Context: ... trusted until the cause is understood. :contentReference[oaicite:18]{index=18} ...
(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~397-~397: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...d size or larger. Why it matters: High power means you are less likely to miss real ...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
[uncategorized] ~397-~397: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ... likely to miss real improvements; very low power leads to many inconclusive or misleadin...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
[formatting] ~415-~415: Insert a comma before quoting reported speech: “states, “”…
Context: ... law Definition: A heuristic that states “the more surprising a result looks, the ...
(SAID_COMMA_SPEECH)
[uncategorized] ~440-~440: Loose punctuation mark.
Context: ...duces how long experiments need to run. :contentReference[oaicite:19]{index=19} ...
(UNLIKELY_OPENING_PUNCTUATION)
[style] ~448-~448: Would you like to use the Oxford spelling “standardized”? The spelling ‘standardised’ is also correct.
Context: ... ## Z ### Z-score Definition: A standardised value expressing how many standard devi...
(OXFORD_SPELLING_Z_NOT_S)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: Redirect rules - absmartly-docs
- GitHub Check: Header rules - absmartly-docs
- GitHub Check: Pages changed - absmartly-docs
- GitHub Check: Yarn Build
🔇 Additional comments (2)
docs/glossary.mdx (2)
1-11: Document structure and front matter are appropriate.The front matter, title, and introductory text are well-formatted and provide good context.
12-450: Content quality and coverage are excellent.The glossary entries are well-structured, clearly written, and provide practical definitions with context. Cross-references are correctly formatted. The comprehensive A–Z coverage supports the PR objective of enhancing experimentation documentation.
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (1)
docs/glossary.mdx (1)
119-119: Remove inline citation artifacts that will render as visible text.The
:contentReference[oaicite:...]{index=...}markers appear 14+ times throughout the glossary and will render as visible pollution in the published Docusaurus site. These are AI-generated content tracking artefacts that serve no purpose in the documentation.Remove all instances to clean up the source. Use a regex search-and-replace such as
/:contentReference\[oaicite:[^\]]*\]\{index:[^\}]*\}/gto strip these markers whilst leaving surrounding sentences and punctuation intact.Apply this example diff to remove one instance (repeat pattern across all 14+ occurrences):
- Continuous monitoring can invalidate fixed-horizon statistical guarantees if no correction is used; sequential methods are designed to handle this safely. :contentReference[oaicite:6]{index=6} + Continuous monitoring can invalidate fixed-horizon statistical guarantees if no correction is used; sequential methods are designed to handle this safely.Also applies to: 134-134, 154-154, 159-159, 188-188, 213-213, 218-218, 223-223, 237-237, 284-284, 354-354, 388-388, 422-422, 475-475
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
docs/glossary.mdx(1 hunks)
🧰 Additional context used
🪛 LanguageTool
docs/glossary.mdx
[grammar] ~15-~15: If ‘type’ is a classification term, ‘A’ is not necessary. Use “type of”. (The phrases ‘kind of’ and ‘sort of’ are informal if they mean ‘to some extent’.)
Context: ... experiment Definition: A special type of A/B experiment where users are randomly s...
(KIND_OF_A)
[style] ~18-~18: Would you like to use the Oxford spelling “randomization”? The spelling ‘randomisation’ is also correct.
Context: ...M), tracking bugs or unexpected bias in randomisation before you start testing real changes. ...
(OXFORD_SPELLING_Z_NOT_S)
[grammar] ~31-~31: If ‘type’ is a classification term, ‘A’ is not necessary. Use “type of”. (The phrases ‘kind of’ and ‘sort of’ are informal if they mean ‘to some extent’.)
Context: ... experiment Definition: A special type of A/B experiment that compares a control to...
(KIND_OF_A)
[style] ~41-~41: Would you like to use the Oxford spelling “generalizability”? The spelling ‘generalisability’ is also correct.
Context: ...eting can reduce sample size and affect generalisability. Example: Running an experiment on...
(OXFORD_SPELLING_Z_NOT_S)
[duplication] ~94-~94: Possible typo: you repeated a word.
Context: ...fidence interval of [+0.5%, +4.1%], it means means that if you were to repeat the same exp...
(ENGLISH_WORD_REPEAT_RULE)
[uncategorized] ~96-~96: Possible missing comma found.
Context: ...the CI in about 95 of those. You could say “We are 95% confident that the true eff...
(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~119-~119: Loose punctuation mark.
Context: ...ods are designed to handle this safely. :contentReference[oaicite:6]{index=6} *...
(UNLIKELY_OPENING_PUNCTUATION)
[grammar] ~133-~133: The verb ‘covariate’ does not usually follow articles like ‘a’. Check that ‘covariate’ is spelled correctly; using ‘covariate’ as a noun may be non-standard.
Context: ...using correlated pre-experiment data as a covariate. Why it matters: CUPED can signif...
(A_INFINITIVE)
[uncategorized] ~134-~134: Use a comma before ‘so’ if it connects two independent clauses (unless they are closely connected and short).
Context: ...ED can significantly improve sensitivity so experiments reach conclusions faster or...
(COMMA_COMPOUND_SENTENCE_2)
[uncategorized] ~134-~134: Loose punctuation mark.
Context: ...smaller effects using the same traffic. :contentReference[oaicite:7]{index=7} *...
(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~141-~141: The noun “decision-making” (= the process of deciding something) is spelled with a hyphen.
Context: ...g the test. --- ## D ### Data-driven decision making Definition: A decision process th...
(DECISION_MAKING)
[uncategorized] ~154-~154: Loose punctuation mark.
Context: ...g with the same experimentation budget. :contentReference[oaicite:8]{index=8} *...
(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~159-~159: Loose punctuation mark.
Context: ...ady clear, preserving traffic and time. :contentReference[oaicite:9]{index=9} *...
(UNLIKELY_OPENING_PUNCTUATION)
[style] ~164-~164: To elevate your writing, try using a synonym here.
Context: ...actions can distort results and make it hard to attribute observed effects to a sing...
(HARD_TO)
[uncategorized] ~169-~169: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...ul changes will show up as significant; low power leads to many missed opportunities. *...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
[style] ~175-~175: Would you like to use the Oxford spelling “generalizes”? The spelling ‘generalises’ is also correct.
Context: ... country site to verify that the effect generalises. ### Exploratory vs confirmatory exper...
(OXFORD_SPELLING_Z_NOT_S)
[style] ~178-~178: Try moving the adverb to make the sentence clearer.
Context: ...irections; confirmatory experiments are designed to rigorously test a specific hypothesis. Why it matters: Mixing the two m...
(SPLIT_INFINITIVE)
[uncategorized] ~188-~188: Loose punctuation mark.
Context: ... rate of wrong “winners” under control. :contentReference[oaicite:10]{index=10} ...
(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~213-~213: Possible missing comma found.
Context: ... horizon methods are conceptually simple but are not robust to unplanned peeking or ...
(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~213-~213: Loose punctuation mark.
Context: ...to unplanned peeking or early stopping. :contentReference[oaicite:11]{index=11} ...
(UNLIKELY_OPENING_PUNCTUATION)
[style] ~214-~214: ‘making a decision’ might be wordy. Consider a shorter alternative.
Context: ...an experiment for exactly two weeks and making a decision only after both weeks have completed. ...
(EN_WORDINESS_PREMIUM_MAKING_A_DECISION)
[uncategorized] ~218-~218: Loose punctuation mark.
Context: ...more complex design and interpretation. :contentReference[oaicite:12]{index=12} ...
(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~223-~223: Loose punctuation mark.
Context: ...be reallocated to more promising ideas. :contentReference[oaicite:13]{index=13} ...
(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~228-~228: Possible missing comma found.
Context: ...her to redesign the experiment, improve metrics or simply move on to other ideas. **E...
(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~236-~236: Specify a number, remove phrase, use “a few”, or use “some”
Context: ...sequential approach where you predefine a small number of checkpoints (interim looks) at which yo...
(SMALL_NUMBER_OF)
[uncategorized] ~237-~237: Loose punctuation mark.
Context: ...here a few well-timed looks are enough. :contentReference[oaicite:14]{index=14} ...
(UNLIKELY_OPENING_PUNCTUATION)
[style] ~250-~250: Would you like to use the Oxford spelling “randomized”? The spelling ‘randomised’ is also correct.
Context: ... anecdotes and observational data up to randomised controlled experiments and meta-analyse...
(OXFORD_SPELLING_Z_NOT_S)
[style] ~251-~251: Would you like to use the Oxford spelling “prioritize”? The spelling ‘prioritise’ is also correct.
Context: ...* The hierarchy of evidence helps teams prioritise decisions based on how reliable and cau...
(OXFORD_SPELLING_Z_NOT_S)
[style] ~265-~265: Would you like to use the Oxford spelling “prioritize”? The spelling ‘prioritise’ is also correct.
Context: ...Why it matters:** Impact estimates help prioritise which experiments to run and inform sam...
(OXFORD_SPELLING_Z_NOT_S)
[uncategorized] ~284-~284: Loose punctuation mark.
Context: ...e and you miss meaningful improvements. :contentReference[oaicite:15]{index=15} ...
(UNLIKELY_OPENING_PUNCTUATION)
[style] ~303-~303: Would you like to use the Oxford spelling “optimize”? The spelling ‘optimise’ is also correct.
Context: ...best. They are useful when you want to optimise several components together, such as he...
(OXFORD_SPELLING_Z_NOT_S)
[formatting] ~304-~304: If the ‘because’ clause is essential to the meaning, do not use a comma before the clause.
Context: ...ntly more traffic than a simple A/B test, because traffic must be spread across many vari...
(COMMA_BEFORE_BECAUSE)
[typographical] ~304-~304: Use a comma after an introductory phrase.
Context: ...tions and the analysis is more complex. For this reason Multivariate experiments are not suppor...
(COMMA_INTRODUCTORY_WORDS_PHRASES)
[style] ~308-~308: Would you like to use the Oxford spelling “optimize”? The spelling ‘optimise’ is also correct.
Context: ... ABsmartly. Example: You want to optimise a landing page with: - 2 different he...
(OXFORD_SPELLING_Z_NOT_S)
[typographical] ~308-~308: Do not use a colon (:) before a series that is introduced by a preposition (‘with’). Remove the colon or add a noun or a noun phrase after the preposition.
Context: ...* You want to optimise a landing page with: - 2 different headlines (H1, H2) - ...
(RP_COLON)
[grammar] ~311-~311: It appears that two hyphens are missing.
Context: ...ero images (I1, I2, I3) - 2 different call to action buttons (C1, C2) A multivariate expe...
(CALL_TO_ACTION_HYPHEN)
[grammar] ~335-~335: Did you mean “sample”?
Context: ...ble from the data, but it is subject to sampling variability. Example: Treatment s...
(IT_IS_JJ_TO_VBG)
[uncategorized] ~340-~340: Possible missing comma found.
Context: ... more powerful for directional questions but must be justified in advance to avoid b...
(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~346-~346: Would you like to use the Oxford spelling “utilization”? The spelling ‘utilisation’ is also correct.
Context: ... Example: Error rate, latency, CPU utilisation or cache hit rate. --- ## P ### Peek...
(OXFORD_SPELLING_Z_NOT_S)
[uncategorized] ~354-~354: Loose punctuation mark.
Context: ...make p-values and intervals unreliable. :contentReference[oaicite:16]{index=16} ...
(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~383-~383: Possible missing comma found.
Context: ...xperimentation is not a one-off activity but a core part of how the organisation bui...
(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~383-~383: Would you like to use the Oxford spelling “organization”? The spelling ‘organisation’ is also correct.
Context: ...off activity but a core part of how the organisation builds products. Example: Marty C...
(OXFORD_SPELLING_Z_NOT_S)
[uncategorized] ~388-~388: Loose punctuation mark.
Context: ... the probability that the null is true. :contentReference[oaicite:17]{index=17} ...
(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~408-~408: Possible missing comma found.
Context: ...oring average order value and page load time while the primary metric is conversion ...
(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~418-~418: Possible missing comma found.
Context: ...* Discounts shown only to the treatment group affecting reference prices for control ...
(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~422-~422: Loose punctuation mark.
Context: ... trusted until the cause is understood. :contentReference[oaicite:18]{index=18} ...
(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~432-~432: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...d size or larger. Why it matters: High power means you are less likely to miss real ...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
[uncategorized] ~432-~432: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ... likely to miss real improvements; very low power leads to many inconclusive or misleadin...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
[formatting] ~450-~450: Insert a comma before quoting reported speech: “states, “”…
Context: ... law Definition: A heuristic that states “the more surprising a result looks, the ...
(SAID_COMMA_SPEECH)
[uncategorized] ~475-~475: Loose punctuation mark.
Context: ...duces how long experiments need to run. :contentReference[oaicite:19]{index=19} ...
(UNLIKELY_OPENING_PUNCTUATION)
[style] ~483-~483: Would you like to use the Oxford spelling “standardized”? The spelling ‘standardised’ is also correct.
Context: ... ## Z ### Z-score Definition: A standardised value expressing how many standard devi...
(OXFORD_SPELLING_Z_NOT_S)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
- GitHub Check: Pages changed - absmartly-docs
- GitHub Check: Yarn Build
There was a problem hiding this comment.
Actionable comments posted: 2
♻️ Duplicate comments (2)
docs/glossary.mdx (2)
265-265: Remove persistent inline citation artefacts from glossary.The
:contentReference[oaicite:...]{index=...}markers still appear throughout this file (lines 265, 270, 275, 289, 336, 407, 443, 477, 532 and others) and will render as visible text in the published site. This issue was flagged in the previous review but remains unresolved.Strip all instances using a global search-and-replace regex (e.g.
/:contentReference\[oaicite:[^\]]*\]\{index:[^\}]*\}/g) or manually delete each marker whilst preserving surrounding punctuation.Also applies to: 270-270, 275-275, 289-289, 336-336, 407-407, 443-443, 477-477, 532-532
93-93: Fix repeated word in confidence interval definition.Line 93 contains the duplicate phrase "it means means". This was flagged in the previous review but remains in the current code.
Apply this diff to remove the repetition:
- it means means that if you were to repeat the same experiment 100 times, the true effect would lie within the CI in about 95 of those. + it means that if you were to repeat the same experiment 100 times, the true effect would lie within the CI in about 95 of those.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
docs/glossary.mdx(1 hunks)
🧰 Additional context used
🪛 LanguageTool
docs/glossary.mdx
[grammar] ~15-~15: If ‘type’ is a classification term, ‘A’ is not necessary. Use “type of”. (The phrases ‘kind of’ and ‘sort of’ are informal if they mean ‘to some extent’.)
Context: ...- ## A ### A/A experiment A special type of A/B experiment where users are randomly s...
(KIND_OF_A)
[style] ~18-~18: Would you like to use the Oxford spelling “randomization”? The spelling ‘randomisation’ is also correct.
Context: ...M), tracking bugs or unexpected bias in randomisation before you start testing real changes. ...
(OXFORD_SPELLING_Z_NOT_S)
[grammar] ~31-~31: If ‘type’ is a classification term, ‘A’ is not necessary. Use “type of”. (The phrases ‘kind of’ and ‘sort of’ are informal if they mean ‘to some extent’.)
Context: ...rate. ### A/B/n experiment A special type of A/B experiment that compares a control to...
(KIND_OF_A)
[style] ~41-~41: Would you like to use the Oxford spelling “generalizability”? The spelling ‘generalisability’ is also correct.
Context: ...eting can reduce sample size and affect generalisability. Example: Running an experiment on...
(OXFORD_SPELLING_Z_NOT_S)
[duplication] ~93-~93: Possible typo: you repeated a word.
Context: ...fidence interval of [+0.5%, +4.1%], it means means that if you were to repeat the same exp...
(ENGLISH_WORD_REPEAT_RULE)
[uncategorized] ~95-~95: Possible missing comma found.
Context: ...the CI in about 95 of those. You could say “We are 95% confident that the [true ef...
(AI_HYDRA_LEO_MISSING_COMMA)
[grammar] ~133-~133: The verb ‘covariate’ does not usually follow articles like ‘a’. Check that ‘covariate’ is spelled correctly; using ‘covariate’ as a noun may be non-standard.
Context: ...using correlated pre-experiment data as a covariate. CUPED can significantly improve sen...
(A_INFINITIVE)
[uncategorized] ~135-~135: Use a comma before ‘so’ if it connects two independent clauses (unless they are closely connected and short).
Context: ...ED can significantly improve sensitivity so experiments reach conclusions faster or...
(COMMA_COMPOUND_SENTENCE_2)
[style] ~166-~166: You’ve already used the word ‘also’ once in your sentence, so using it again may be redundant.
Context: ... but also practically meaningful. It's also used in power calculations when designing ex...
(REDUNDANT_FILLER)
[style] ~180-~180: To elevate your writing, try using a synonym here.
Context: ...actions can distort results and make it hard to attribute observed effects to a sing...
(HARD_TO)
[uncategorized] ~192-~192: Possible missing comma found.
Context: ...ngful changes or underestimating effect sizes which leads to unreliable decisions. A...
(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~208-~208: Try moving the adverb to make the sentence clearer.
Context: ...irections; confirmatory experiments are designed to rigorously test a specific hypothesis. Mixing the two modes can lead to inf...
(SPLIT_INFINITIVE)
[grammar] ~230-~230: After ‘it’, use the third-person verb form “exists”.
Context: ...Failing to detect a real effect when it exist; equivalent to a Type II error. False...
(IT_VBZ)
[uncategorized] ~265-~265: Possible missing comma found.
Context: ... horizon methods are conceptually simple but are not robust to unplanned peeking or ...
(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~265-~265: Loose punctuation mark.
Context: ...to unplanned peeking or early stopping. :contentReference[oaicite:11]{index=11} ...
(UNLIKELY_OPENING_PUNCTUATION)
[style] ~266-~266: ‘making a decision’ might be wordy. Consider a shorter alternative.
Context: ...an experiment for exactly two weeks and making a decision only after both weeks have completed. ...
(EN_WORDINESS_PREMIUM_MAKING_A_DECISION)
[uncategorized] ~270-~270: Loose punctuation mark.
Context: ...more complex design and interpretation. :contentReference[oaicite:12]{index=12} ...
(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~275-~275: Loose punctuation mark.
Context: ...be reallocated to more promising ideas. :contentReference[oaicite:13]{index=13} ...
(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~280-~280: Possible missing comma found.
Context: ...her to redesign the experiment, improve metrics or simply move on to other ideas. **E...
(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~288-~288: Specify a number, remove phrase, use “a few”, or use “some”
Context: ...sequential approach where you predefine a small number of checkpoints (interim looks) at which yo...
(SMALL_NUMBER_OF)
[uncategorized] ~289-~289: Loose punctuation mark.
Context: ...here a few well-timed looks are enough. :contentReference[oaicite:14]{index=14} ...
(UNLIKELY_OPENING_PUNCTUATION)
[duplication] ~292-~292: Possible typo: you repeated a word.
Context: ...mple has been collected. ### Guardrail metrics Metrics monitored to ensure experiments stay wi...
(ENGLISH_WORD_REPEAT_RULE)
[style] ~302-~302: Would you like to use the Oxford spelling “randomized”? The spelling ‘randomised’ is also correct.
Context: ... anecdotes and observational data up to randomised controlled experiments and meta-analyse...
(OXFORD_SPELLING_Z_NOT_S)
[style] ~303-~303: Would you like to use the Oxford spelling “prioritize”? The spelling ‘prioritise’ is also correct.
Context: ... The hierarchy of evidence helps teams prioritise decisions based on how reliable and cau...
(OXFORD_SPELLING_Z_NOT_S)
[style] ~317-~317: Would you like to use the Oxford spelling “prioritize”? The spelling ‘prioritise’ is also correct.
Context: ...pert judgement. Impact estimates help prioritise which experiments to run and inform sam...
(OXFORD_SPELLING_Z_NOT_S)
[uncategorized] ~336-~336: Loose punctuation mark.
Context: ...e and you miss meaningful improvements. :contentReference[oaicite:15]{index=15} ...
(UNLIKELY_OPENING_PUNCTUATION)
[style] ~355-~355: Would you like to use the Oxford spelling “optimize”? The spelling ‘optimise’ is also correct.
Context: ...best. They are useful when you want to optimise several components together, such as he...
(OXFORD_SPELLING_Z_NOT_S)
[formatting] ~356-~356: If the ‘because’ clause is essential to the meaning, do not use a comma before the clause.
Context: ...ntly more traffic than a simple A/B test, because traffic must be spread across many vari...
(COMMA_BEFORE_BECAUSE)
[typographical] ~356-~356: Use a comma after an introductory phrase.
Context: ...tions and the analysis is more complex. For this reason Multivariate experiments are not suppor...
(COMMA_INTRODUCTORY_WORDS_PHRASES)
[style] ~360-~360: Would you like to use the Oxford spelling “optimize”? The spelling ‘optimise’ is also correct.
Context: ... ABsmartly. Example: You want to optimise a landing page with: - 2 different he...
(OXFORD_SPELLING_Z_NOT_S)
[typographical] ~360-~360: Do not use a colon (:) before a series that is introduced by a preposition (‘with’). Remove the colon or add a noun or a noun phrase after the preposition.
Context: ...* You want to optimise a landing page with: - 2 different headlines (H1, H2) - ...
(RP_COLON)
[grammar] ~363-~363: It appears that two hyphens are missing.
Context: ...ero images (I1, I2, I3) - 2 different call to action buttons (C1, C2) A multivariate expe...
(CALL_TO_ACTION_HYPHEN)
[uncategorized] ~385-~385: Possible missing preposition found.
Context: ...low.” --- ## O ### Observed effect The measured difference between treatment a...
(AI_HYDRA_LEO_MISSING_IN)
[grammar] ~387-~387: Did you mean “sample”?
Context: ...ble from the data, but it is subject to sampling variability. Example: Treatment ...
(IT_IS_JJ_TO_VBG)
[uncategorized] ~393-~393: Possible missing comma found.
Context: ... more powerful for directional questions but must be justified in advance to avoid b...
(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~399-~399: Would you like to use the Oxford spelling “utilization”? The spelling ‘utilisation’ is also correct.
Context: ... Example: Error rate, latency, CPU utilisation or cache hit rate. --- ## P ### Peek...
(OXFORD_SPELLING_Z_NOT_S)
[uncategorized] ~407-~407: Loose punctuation mark.
Context: ...make p-values and intervals unreliable. :contentReference[oaicite:16]{index=16} ...
(UNLIKELY_OPENING_PUNCTUATION)
[duplication] ~421-~421: Possible typo: you repeated a word.
Context: ... significance level. ### Pre-selection bias Bias introduced when the users who enter a s...
(ENGLISH_WORD_REPEAT_RULE)
[uncategorized] ~438-~438: Possible missing comma found.
Context: ...xperimentation is not a one-off activity but a core part of how the organisation bui...
(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~438-~438: Do not mix variants of the same word (‘organisation’ and ‘organization’) within a single text.
Context: ...off activity but a core part of how the organisation builds products. Example: Marty C...
(EN_WORD_COHERENCY)
[uncategorized] ~443-~443: Loose punctuation mark.
Context: ... the probability that the null is true. :contentReference[oaicite:17]{index=17} ...
(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~463-~463: Possible missing comma found.
Context: ...oring average order value and page load time while the primary metric is conversion ...
(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~473-~473: Possible missing comma found.
Context: ...* Discounts shown only to the treatment group affecting reference prices for control ...
(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~477-~477: Loose punctuation mark.
Context: ... trusted until the cause is understood. :contentReference[oaicite:18]{index=18} ...
(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~486-~486: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...effect of the planned size or larger. High power means you are less likely to miss real ...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
[uncategorized] ~487-~487: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ... likely to miss real improvements; very low power leads to many inconclusive or misleadin...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
[formatting] ~507-~507: Insert a comma before quoting reported speech: “states, “”…
Context: ...t. ### Twyman’s law A heuristic that states “the more surprising a result looks, the ...
(SAID_COMMA_SPEECH)
[uncategorized] ~532-~532: Loose punctuation mark.
Context: ...duces how long experiments need to run. :contentReference[oaicite:19]{index=19} ...
(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~540-~540: Do not mix variants of the same word (‘standardise’ and ‘standardize’) within a single text.
Context: ...me traffic. --- ## Z ### Z-score A standardised value expressing how many standard devi...
(EN_WORD_COHERENCY)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: Yarn Build
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (3)
docs/glossary.mdx (3)
93-93: Fix duplicated word at line 93."It means means" contains a repeated word. This appears despite a prior review comment noting the same issue.
Apply this diff:
-it means means that if you were to repeat the same experiment 100 times, the true effect would lie within the CI in about 95 of those. +it means that if you were to repeat the same experiment 100 times, the true effect would lie within the CI in about 95 of those.
230-230: Fix subject–verb agreement in false negative definition."When it exist" is missing the third-person singular verb form. Should read "when it exists".
Apply this diff:
-Failing to detect a real effect when it exist; equivalent to a Type II error. +Failing to detect a real effect when it exists; equivalent to a Type II error.
387-387: Add hyphen to compound modifier in multivariate experiment example."Call to action buttons" is used as a compound modifier and requires hyphens: "call-to-action buttons".
Apply this diff:
-- 2 different call to action buttons (C1, C2) +- 2 different call-to-action buttons (C1, C2)
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
docs/glossary.mdx(1 hunks)
🧰 Additional context used
🪛 LanguageTool
docs/glossary.mdx
[grammar] ~15-~15: If ‘type’ is a classification term, ‘A’ is not necessary. Use “type of”. (The phrases ‘kind of’ and ‘sort of’ are informal if they mean ‘to some extent’.)
Context: ...- ## A ### A/A experiment A special type of A/B experiment where users are randomly s...
(KIND_OF_A)
[style] ~18-~18: Would you like to use the Oxford spelling “randomization”? The spelling ‘randomisation’ is also correct.
Context: ...M), tracking bugs or unexpected bias in randomisation before you start testing real changes. ...
(OXFORD_SPELLING_Z_NOT_S)
[grammar] ~31-~31: If ‘type’ is a classification term, ‘A’ is not necessary. Use “type of”. (The phrases ‘kind of’ and ‘sort of’ are informal if they mean ‘to some extent’.)
Context: ...rate. ### A/B/n experiment A special type of A/B experiment that compares a control to...
(KIND_OF_A)
[style] ~41-~41: Would you like to use the Oxford spelling “generalizability”? The spelling ‘generalisability’ is also correct.
Context: ...eting can reduce sample size and affect generalisability. Example: Running an experiment on...
(OXFORD_SPELLING_Z_NOT_S)
[duplication] ~93-~93: Possible typo: you repeated a word.
Context: ...fidence interval of [+0.5%, +4.1%], it means means that if you were to repeat the same exp...
(ENGLISH_WORD_REPEAT_RULE)
[uncategorized] ~95-~95: Possible missing comma found.
Context: ...the CI in about 95 of those. You could say “We are 95% confident that the [true ef...
(AI_HYDRA_LEO_MISSING_COMMA)
[grammar] ~133-~133: The verb ‘covariate’ does not usually follow articles like ‘a’. Check that ‘covariate’ is spelled correctly; using ‘covariate’ as a noun may be non-standard.
Context: ...using correlated pre-experiment data as a covariate. CUPED can significantly improve sen...
(A_INFINITIVE)
[uncategorized] ~135-~135: Use a comma before ‘so’ if it connects two independent clauses (unless they are closely connected and short).
Context: ...ED can significantly improve sensitivity so experiments reach conclusions faster or...
(COMMA_COMPOUND_SENTENCE_2)
[style] ~166-~166: You’ve already used the word ‘also’ once in your sentence, so using it again may be redundant.
Context: ... but also practically meaningful. It's also used in power calculations when designing ex...
(REDUNDANT_FILLER)
[style] ~180-~180: To elevate your writing, try using a synonym here.
Context: ...actions can distort results and make it hard to attribute observed effects to a sing...
(HARD_TO)
[uncategorized] ~192-~192: Possible missing comma found.
Context: ...ngful changes or underestimating effect sizes which leads to unreliable decisions. A...
(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~208-~208: Try moving the adverb to make the sentence clearer.
Context: ...irections; confirmatory experiments are designed to rigorously test a specific hypothesis. Mixing the two modes can lead to inf...
(SPLIT_INFINITIVE)
[grammar] ~230-~230: After ‘it’, use the third-person verb form “exists”.
Context: ...Failing to detect a real effect when it exist; equivalent to a Type II error. False...
(IT_VBZ)
[uncategorized] ~268-~268: Possible missing comma found.
Context: ... horizon methods are conceptually simple but are not robust to unplanned peeking or ...
(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~272-~272: ‘making a decision’ might be wordy. Consider a shorter alternative.
Context: ...an experiment for exactly two weeks and making a decision only after both weeks have completed. ...
(EN_WORDINESS_PREMIUM_MAKING_A_DECISION)
[style] ~282-~282: ‘for lack of’ might be wordy. Consider a shorter alternative.
Context: ...is, allows the experiment to stop early for lack of effect — indicating that the treatment ...
(EN_WORDINESS_PREMIUM_FOR_LACK_OF)
[uncategorized] ~300-~300: Did you mean: “By default,”?
Context: ...ces stricter control over error rates. By default GST experiment in ABsmartly uses a bind...
(BY_DEFAULT_COMMA)
[uncategorized] ~301-~301: Use a comma before ‘but’ if it connects two independent clauses (unless they are closely connected and short).
Context: ...n ABsmartly uses a binding futility type but this can be changed during the setup. ...
(COMMA_COMPOUND_SENTENCE_2)
[style] ~312-~312: Specify a number, remove phrase, use “a few”, or use “some”
Context: ...sequential approach where you predefine a small number of checkpoints (interim looks) at which yo...
(SMALL_NUMBER_OF)
[uncategorized] ~313-~313: Loose punctuation mark.
Context: ...here a few well-timed looks are enough. :contentReference[oaicite:14]{index=14} ...
(UNLIKELY_OPENING_PUNCTUATION)
[duplication] ~316-~316: Possible typo: you repeated a word.
Context: ...mple has been collected. ### Guardrail metrics Metrics monitored to ensure experiments stay wi...
(ENGLISH_WORD_REPEAT_RULE)
[style] ~326-~326: Would you like to use the Oxford spelling “randomized”? The spelling ‘randomised’ is also correct.
Context: ... anecdotes and observational data up to randomised controlled experiments and meta-analyse...
(OXFORD_SPELLING_Z_NOT_S)
[style] ~327-~327: Would you like to use the Oxford spelling “prioritize”? The spelling ‘prioritise’ is also correct.
Context: ... The hierarchy of evidence helps teams prioritise decisions based on how reliable and cau...
(OXFORD_SPELLING_Z_NOT_S)
[uncategorized] ~340-~340: Do not mix variants of the same word (‘judgement’ and ‘judgment’) within a single text.
Context: ...ased on prior data, modelling or expert judgement. Impact estimates help prioritise whi...
(EN_WORD_COHERENCY)
[style] ~341-~341: Would you like to use the Oxford spelling “prioritize”? The spelling ‘prioritise’ is also correct.
Context: ...pert judgement. Impact estimates help prioritise which experiments to run and inform sam...
(OXFORD_SPELLING_Z_NOT_S)
[uncategorized] ~360-~360: Loose punctuation mark.
Context: ...e and you miss meaningful improvements. :contentReference[oaicite:15]{index=15} ...
(UNLIKELY_OPENING_PUNCTUATION)
[style] ~379-~379: Would you like to use the Oxford spelling “optimize”? The spelling ‘optimise’ is also correct.
Context: ...best. They are useful when you want to optimise several components together, such as he...
(OXFORD_SPELLING_Z_NOT_S)
[formatting] ~380-~380: If the ‘because’ clause is essential to the meaning, do not use a comma before the clause.
Context: ...ntly more traffic than a simple A/B test, because traffic must be spread across many vari...
(COMMA_BEFORE_BECAUSE)
[typographical] ~380-~380: Use a comma after an introductory phrase.
Context: ...tions and the analysis is more complex. For this reason Multivariate experiments are not suppor...
(COMMA_INTRODUCTORY_WORDS_PHRASES)
[style] ~384-~384: Would you like to use the Oxford spelling “optimize”? The spelling ‘optimise’ is also correct.
Context: ... ABsmartly. Example: You want to optimise a landing page with: - 2 different he...
(OXFORD_SPELLING_Z_NOT_S)
[typographical] ~384-~384: Do not use a colon (:) before a series that is introduced by a preposition (‘with’). Remove the colon or add a noun or a noun phrase after the preposition.
Context: ...* You want to optimise a landing page with: - 2 different headlines (H1, H2) - ...
(RP_COLON)
[grammar] ~387-~387: It appears that two hyphens are missing.
Context: ...ero images (I1, I2, I3) - 2 different call to action buttons (C1, C2) A multivariate expe...
(CALL_TO_ACTION_HYPHEN)
[uncategorized] ~409-~409: Possible missing preposition found.
Context: ...low.” --- ## O ### Observed effect The measured difference between treatment a...
(AI_HYDRA_LEO_MISSING_IN)
[grammar] ~411-~411: Did you mean “sample”?
Context: ...ble from the data, but it is subject to sampling variability. Example: Treatment ...
(IT_IS_JJ_TO_VBG)
[uncategorized] ~417-~417: Possible missing comma found.
Context: ... more powerful for directional questions but must be justified in advance to avoid b...
(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~423-~423: Would you like to use the Oxford spelling “utilization”? The spelling ‘utilisation’ is also correct.
Context: ... Example: Error rate, latency, CPU utilisation or cache hit rate. --- ## P ### Peek...
(OXFORD_SPELLING_Z_NOT_S)
[uncategorized] ~431-~431: Loose punctuation mark.
Context: ...make p-values and intervals unreliable. :contentReference[oaicite:16]{index=16} ...
(UNLIKELY_OPENING_PUNCTUATION)
[duplication] ~445-~445: Possible typo: you repeated a word.
Context: ... significance level. ### Pre-selection bias Bias introduced when the users who enter a s...
(ENGLISH_WORD_REPEAT_RULE)
[uncategorized] ~462-~462: Possible missing comma found.
Context: ...xperimentation is not a one-off activity but a core part of how the organisation bui...
(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~462-~462: Do not mix variants of the same word (‘organisation’ and ‘organization’) within a single text.
Context: ...off activity but a core part of how the organisation builds products. Example: Marty C...
(EN_WORD_COHERENCY)
[uncategorized] ~467-~467: Loose punctuation mark.
Context: ... the probability that the null is true. :contentReference[oaicite:17]{index=17} ...
(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~487-~487: Possible missing comma found.
Context: ...oring average order value and page load time while the primary metric is conversion ...
(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~497-~497: Possible missing comma found.
Context: ...* Discounts shown only to the treatment group affecting reference prices for control ...
(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~501-~501: Loose punctuation mark.
Context: ... trusted until the cause is understood. :contentReference[oaicite:18]{index=18} ...
(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~510-~510: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...effect of the planned size or larger. High power means you are less likely to miss real ...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
[uncategorized] ~511-~511: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ... likely to miss real improvements; very low power leads to many inconclusive or misleadin...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
[formatting] ~531-~531: Insert a comma before quoting reported speech: “states, “”…
Context: ...t. ### Twyman’s law A heuristic that states “the more surprising a result looks, the ...
(SAID_COMMA_SPEECH)
[uncategorized] ~556-~556: Loose punctuation mark.
Context: ...duces how long experiments need to run. :contentReference[oaicite:19]{index=19} ...
(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~564-~564: Do not mix variants of the same word (‘standardise’ and ‘standardize’) within a single text.
Context: ...me traffic. --- ## Z ### Z-score A standardised value expressing how many standard devi...
(EN_WORD_COHERENCY)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: Redirect rules - absmartly-docs
- GitHub Check: Header rules - absmartly-docs
- GitHub Check: Pages changed - absmartly-docs
- GitHub Check: Yarn Build
There was a problem hiding this comment.
Actionable comments posted: 4
♻️ Duplicate comments (4)
docs/glossary.mdx (4)
364-364: Remove inline citation artefacts that were flagged in prior reviews.Past review comments (marked as addressed) flagged inline
:contentReference[oaicite:...]{index=...}markers that will render as visible artefacts in published documentation. These markers remain present at lines 364, 435, 471, and 505 (and potentially line 560 per static analysis). Remove all occurrences using a global search-and-replace to eliminate these AI-tracking markers whilst preserving surrounding text and punctuation.Apply this regex pattern to remove all instances:
:contentReference\[oaicite:[^\]]*\]\{index=[^\}]*\}Expected occurrences to remove:
- Line 364:
:contentReference[oaicite:15]{index=15}- Line 435:
:contentReference[oaicite:16]{index=16}- Line 471:
:contentReference[oaicite:17]{index=17}- Line 505:
:contentReference[oaicite:18]{index=18}- Line 560:
:contentReference[oaicite:19]{index=19}Also applies to: 435-435, 471-471, 505-505
93-93: Fix word repetition typo flagged in prior review.Line 93 contains duplicated word "means": "it means means that". This was previously flagged as resolved but remains in the current code.
Apply this diff:
- it means means that if you were to repeat the same experiment 100 times, the true effect would lie within the CI in about 95 of those. + it means that if you were to repeat the same experiment 100 times, the true effect would lie within the CI in about 95 of those.
230-230: Fix subject–verb agreement flagged in prior review.Line 230 contains grammatical error: "when it exist" should use the third-person verb form "exists".
Apply this diff:
- Failing to detect a real effect when it exist; equivalent to a Type II error. + Failing to detect a real effect when it exists; equivalent to a Type II error.
391-391: Add hyphens to compound modifier flagged in prior review.Line 391 uses "call to action buttons" as a compound modifier and requires hyphenation: "call-to-action buttons".
Apply this diff:
- - 2 different call to action buttons (C1, C2) + - 2 different call-to-action buttons (C1, C2)
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
docs/glossary.mdx(1 hunks)
🧰 Additional context used
🪛 LanguageTool
docs/glossary.mdx
[grammar] ~15-~15: If ‘type’ is a classification term, ‘A’ is not necessary. Use “type of”. (The phrases ‘kind of’ and ‘sort of’ are informal if they mean ‘to some extent’.)
Context: ...- ## A ### A/A experiment A special type of A/B experiment where users are randomly s...
(KIND_OF_A)
[style] ~18-~18: Would you like to use the Oxford spelling “randomization”? The spelling ‘randomisation’ is also correct.
Context: ...M), tracking bugs or unexpected bias in randomisation before you start testing real changes. ...
(OXFORD_SPELLING_Z_NOT_S)
[grammar] ~31-~31: If ‘type’ is a classification term, ‘A’ is not necessary. Use “type of”. (The phrases ‘kind of’ and ‘sort of’ are informal if they mean ‘to some extent’.)
Context: ...rate. ### A/B/n experiment A special type of A/B experiment that compares a control to...
(KIND_OF_A)
[style] ~41-~41: Would you like to use the Oxford spelling “generalizability”? The spelling ‘generalisability’ is also correct.
Context: ...eting can reduce sample size and affect generalisability. Example: Running an experiment on...
(OXFORD_SPELLING_Z_NOT_S)
[duplication] ~93-~93: Possible typo: you repeated a word.
Context: ...fidence interval of [+0.5%, +4.1%], it means means that if you were to repeat the same exp...
(ENGLISH_WORD_REPEAT_RULE)
[uncategorized] ~95-~95: Possible missing comma found.
Context: ...the CI in about 95 of those. You could say “We are 95% confident that the [true ef...
(AI_HYDRA_LEO_MISSING_COMMA)
[grammar] ~133-~133: The verb ‘covariate’ does not usually follow articles like ‘a’. Check that ‘covariate’ is spelled correctly; using ‘covariate’ as a noun may be non-standard.
Context: ...using correlated pre-experiment data as a covariate. CUPED can significantly improve sen...
(A_INFINITIVE)
[uncategorized] ~135-~135: Use a comma before ‘so’ if it connects two independent clauses (unless they are closely connected and short).
Context: ...ED can significantly improve sensitivity so experiments reach conclusions faster or...
(COMMA_COMPOUND_SENTENCE_2)
[style] ~166-~166: You’ve already used the word ‘also’ once in your sentence, so using it again may be redundant.
Context: ... but also practically meaningful. It's also used in power calculations when designing ex...
(REDUNDANT_FILLER)
[style] ~180-~180: To elevate your writing, try using a synonym here.
Context: ...actions can distort results and make it hard to attribute observed effects to a sing...
(HARD_TO)
[uncategorized] ~192-~192: Possible missing comma found.
Context: ...ngful changes or underestimating effect sizes which leads to unreliable decisions. A...
(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~208-~208: Try moving the adverb to make the sentence clearer.
Context: ...irections; confirmatory experiments are designed to rigorously test a specific hypothesis. Mixing the two modes can lead to inf...
(SPLIT_INFINITIVE)
[grammar] ~230-~230: After ‘it’, use the third-person verb form “exists”.
Context: ...Failing to detect a real effect when it exist; equivalent to a Type II error. False...
(IT_VBZ)
[uncategorized] ~268-~268: Possible missing comma found.
Context: ... horizon methods are conceptually simple but are not robust to unplanned peeking or ...
(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~272-~272: ‘making a decision’ might be wordy. Consider a shorter alternative.
Context: ...an experiment for exactly two weeks and making a decision only after both weeks have completed. ...
(EN_WORDINESS_PREMIUM_MAKING_A_DECISION)
[style] ~282-~282: ‘for lack of’ might be wordy. Consider a shorter alternative.
Context: ...is, allows the experiment to stop early for lack of effect — indicating that the treatment ...
(EN_WORDINESS_PREMIUM_FOR_LACK_OF)
[uncategorized] ~300-~300: Did you mean: “By default,”?
Context: ...ces stricter control over error rates. By default GST experiment in ABsmartly uses a bind...
(BY_DEFAULT_COMMA)
[uncategorized] ~301-~301: Use a comma before ‘but’ if it connects two independent clauses (unless they are closely connected and short).
Context: ...n ABsmartly uses a binding futility type but this can be changed during the setup. ...
(COMMA_COMPOUND_SENTENCE_2)
[style] ~314-~314: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...re a few well-timed looks are enough. Group Sequential Testing is the default metho...
(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
[duplication] ~320-~320: Possible typo: you repeated a word.
Context: ...ps://absmartly.com/gst') ### Guardrail metrics Metrics monitored to ensure experiments stay wi...
(ENGLISH_WORD_REPEAT_RULE)
[style] ~330-~330: Would you like to use the Oxford spelling “randomized”? The spelling ‘randomised’ is also correct.
Context: ... anecdotes and observational data up to randomised controlled experiments and meta-analyse...
(OXFORD_SPELLING_Z_NOT_S)
[style] ~331-~331: Would you like to use the Oxford spelling “prioritize”? The spelling ‘prioritise’ is also correct.
Context: ... The hierarchy of evidence helps teams prioritise decisions based on how reliable and cau...
(OXFORD_SPELLING_Z_NOT_S)
[uncategorized] ~344-~344: Do not mix variants of the same word (‘judgement’ and ‘judgment’) within a single text.
Context: ...ased on prior data, modelling or expert judgement. Impact estimates help prioritise whi...
(EN_WORD_COHERENCY)
[style] ~345-~345: Would you like to use the Oxford spelling “prioritize”? The spelling ‘prioritise’ is also correct.
Context: ...pert judgement. Impact estimates help prioritise which experiments to run and inform sam...
(OXFORD_SPELLING_Z_NOT_S)
[uncategorized] ~364-~364: Loose punctuation mark.
Context: ...e and you miss meaningful improvements. :contentReference[oaicite:15]{index=15} ...
(UNLIKELY_OPENING_PUNCTUATION)
[style] ~383-~383: Would you like to use the Oxford spelling “optimize”? The spelling ‘optimise’ is also correct.
Context: ...best. They are useful when you want to optimise several components together, such as he...
(OXFORD_SPELLING_Z_NOT_S)
[formatting] ~384-~384: If the ‘because’ clause is essential to the meaning, do not use a comma before the clause.
Context: ...ntly more traffic than a simple A/B test, because traffic must be spread across many vari...
(COMMA_BEFORE_BECAUSE)
[typographical] ~384-~384: Use a comma after an introductory phrase.
Context: ...tions and the analysis is more complex. For this reason Multivariate experiments are not suppor...
(COMMA_INTRODUCTORY_WORDS_PHRASES)
[style] ~388-~388: Would you like to use the Oxford spelling “optimize”? The spelling ‘optimise’ is also correct.
Context: ... ABsmartly. Example: You want to optimise a landing page with: - 2 different he...
(OXFORD_SPELLING_Z_NOT_S)
[typographical] ~388-~388: Do not use a colon (:) before a series that is introduced by a preposition (‘with’). Remove the colon or add a noun or a noun phrase after the preposition.
Context: ...* You want to optimise a landing page with: - 2 different headlines (H1, H2) - ...
(RP_COLON)
[grammar] ~391-~391: It appears that two hyphens are missing.
Context: ...ero images (I1, I2, I3) - 2 different call to action buttons (C1, C2) A multivariate expe...
(CALL_TO_ACTION_HYPHEN)
[uncategorized] ~413-~413: Possible missing preposition found.
Context: ...low.” --- ## O ### Observed effect The measured difference between treatment a...
(AI_HYDRA_LEO_MISSING_IN)
[grammar] ~415-~415: Did you mean “sample”?
Context: ...ble from the data, but it is subject to sampling variability. Example: Treatment ...
(IT_IS_JJ_TO_VBG)
[uncategorized] ~421-~421: Possible missing comma found.
Context: ... more powerful for directional questions but must be justified in advance to avoid b...
(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~427-~427: Would you like to use the Oxford spelling “utilization”? The spelling ‘utilisation’ is also correct.
Context: ... Example: Error rate, latency, CPU utilisation or cache hit rate. --- ## P ### Peek...
(OXFORD_SPELLING_Z_NOT_S)
[uncategorized] ~435-~435: Loose punctuation mark.
Context: ...make p-values and intervals unreliable. :contentReference[oaicite:16]{index=16} ...
(UNLIKELY_OPENING_PUNCTUATION)
[duplication] ~449-~449: Possible typo: you repeated a word.
Context: ... significance level. ### Pre-selection bias Bias introduced when the users who enter a s...
(ENGLISH_WORD_REPEAT_RULE)
[uncategorized] ~466-~466: Possible missing comma found.
Context: ...xperimentation is not a one-off activity but a core part of how the organisation bui...
(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~466-~466: Do not mix variants of the same word (‘organisation’ and ‘organization’) within a single text.
Context: ...off activity but a core part of how the organisation builds products. Example: Marty C...
(EN_WORD_COHERENCY)
[uncategorized] ~471-~471: Loose punctuation mark.
Context: ... the probability that the null is true. :contentReference[oaicite:17]{index=17} ...
(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~491-~491: Possible missing comma found.
Context: ...oring average order value and page load time while the primary metric is conversion ...
(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~501-~501: Possible missing comma found.
Context: ...* Discounts shown only to the treatment group affecting reference prices for control ...
(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~505-~505: Loose punctuation mark.
Context: ... trusted until the cause is understood. :contentReference[oaicite:18]{index=18} ...
(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~514-~514: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...effect of the planned size or larger. High power means you are less likely to miss real ...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
[uncategorized] ~515-~515: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ... likely to miss real improvements; very low power leads to many inconclusive or misleadin...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
[formatting] ~535-~535: Insert a comma before quoting reported speech: “states, “”…
Context: ...t. ### Twyman’s law A heuristic that states “the more surprising a result looks, the ...
(SAID_COMMA_SPEECH)
[uncategorized] ~560-~560: Loose punctuation mark.
Context: ...duces how long experiments need to run. :contentReference[oaicite:19]{index=19} ...
(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~568-~568: Do not mix variants of the same word (‘standardise’ and ‘standardize’) within a single text.
Context: ...me traffic. --- ## Z ### Z-score A standardised value expressing how many standard devi...
(EN_WORD_COHERENCY)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: Yarn Build
There was a problem hiding this comment.
Actionable comments posted: 0
♻️ Duplicate comments (5)
docs/glossary.mdx (5)
32-32: Fix spelling error in A/B/n definition.Line 32 contains a misspelling: "refered" should be "referred".
Apply this diff:
-A/B/n experiments are also sometimes refered to as multi-variant. Not to be confused with [Multivariate Experiment](#multivariate-experiment). +A/B/n experiments are also sometimes referred to as multi-variant. Not to be confused with [Multivariate Experiment](#multivariate-experiment).
92-92: Fix word repetition and add missing comma in confidence interval example.Line 92 contains a duplicated word ("it means means"), and line 94 needs a comma after the closing quotation mark before the independent clause.
Apply this diff:
-it means means that if you were to repeat the same experiment 100 times, the true effect would lie within the CI in about 95 of those. +it means that if you were to repeat the same experiment 100 times, the true effect would lie within the CI in about 95 of those. -You could say "We are 95% confident that the [true effect](#true-effect) of the treatment is between +0.5% and +4.1%." +You could say, "We are 95% confident that the [true effect](#true-effect) of the treatment is between +0.5% and +4.1%."Also applies to: 94-94
246-246: Fix subject–verb agreement in false negative definition.Line 246 reads "when it exist", but the subject is third-person singular and requires "exists".
Apply this diff:
-Failing to detect a real effect when it exist; equivalent to a Type II error. +Failing to detect a real effect when it exists; equivalent to a Type II error.
316-316: Add missing commas in GST futility type explanation.Lines 316–317 need two commas: one after "By default" and one before "but" when joining two independent clauses.
Apply this diff:
-By default GST experiment in ABsmartly uses a binding futility type but this can be changed during the setup. +By default, GST experiment in ABsmartly uses a binding futility type, but this can be changed during the setup.Also applies to: 317-317
403-403: Remove colon after preposition and add hyphens to compound modifier.Line 403 uses a colon after the preposition "with" (grammatically incorrect), and line 406 omits hyphens in the compound adjective "call to action buttons".
Apply this diff:
-You want to optimise a landing page with: - -- 2 different headlines (H1, H2) -- 3 different hero images (I1, I2, I3) -- 2 different call to action buttons (C1, C2) +You want to optimise a landing page with the following: + +- 2 different headlines (H1, H2) +- 3 different hero images (I1, I2, I3) +- 2 different call-to-action buttons (C1, C2)Also applies to: 406-406
🧹 Nitpick comments (3)
docs/glossary.mdx (3)
165-165: Remove redundant use of "also" in Effect size definition.Line 165 uses "also" twice in close proximity within the same explanation, which is repetitive.
Apply this diff:
-Effect size is essential for determining whether a result is not just statistically significant, but also practically meaningful. -It's also used in power calculations when designing experiments. +Effect size is essential for determining whether a result is not just statistically significant but also practically meaningful. +It is used in power calculations when designing experiments.
329-329: Improve sentence variety in Group sequential testing section.Lines 329–333 are three successive sentences opening with "Group Sequential Testing" or similar phrasing, which reduces readability.
Consider rewording at least one sentence for variety:
-Group Sequential Testing is the default method when creating new experiment. -It leads to making decisions up to 80% than with a more traditional [Fixed Horizon Experiment](#fixed-horizon-testing). +When creating new experiments, Group Sequential Testing is the default method. +It can accelerate decisions by up to 80% compared to more traditional [Fixed Horizon Experiment](#fixed-horizon-testing).Also applies to: 333-333
481-481: Fix inconsistent British spelling (organisation vs organization).Line 481 uses "organisation" while other parts of the glossary use American spelling. For consistency across the entire file, standardise on one variant.
Review the entire file and choose either British (organisation, optimise, prioritise, standardise) or American (organization, optimize, prioritize, standardize) spelling throughout. This affects multiple entries (lines 41, 360, 398, 403, 442, 481, 583).
For example:
-how the organisation builds products. +how the organization builds products.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
docs/glossary.mdx(1 hunks)
🧰 Additional context used
🪛 LanguageTool
docs/glossary.mdx
[grammar] ~15-~15: If ‘type’ is a classification term, ‘A’ is not necessary. Use “type of”. (The phrases ‘kind of’ and ‘sort of’ are informal if they mean ‘to some extent’.)
Context: ...- ## A ### A/A experiment A special type of A/B experiment where users are randomly s...
(KIND_OF_A)
[style] ~18-~18: Would you like to use the Oxford spelling “randomization”? The spelling ‘randomisation’ is also correct.
Context: ...M), tracking bugs or unexpected bias in randomisation before you start testing real changes. ...
(OXFORD_SPELLING_Z_NOT_S)
[grammar] ~31-~31: If ‘type’ is a classification term, ‘A’ is not necessary. Use “type of”. (The phrases ‘kind of’ and ‘sort of’ are informal if they mean ‘to some extent’.)
Context: ...rate. ### A/B/n experiment A special type of A/B experiment that compares a control to...
(KIND_OF_A)
[style] ~41-~41: Would you like to use the Oxford spelling “generalizability”? The spelling ‘generalisability’ is also correct.
Context: ...eting can reduce sample size and affect generalisability. Example: Running an experiment on...
(OXFORD_SPELLING_Z_NOT_S)
[duplication] ~92-~92: Possible typo: you repeated a word.
Context: ...fidence interval of [+0.5%, +4.1%], it means means that if you were to repeat the same exp...
(ENGLISH_WORD_REPEAT_RULE)
[uncategorized] ~94-~94: Possible missing comma found.
Context: ...the CI in about 95 of those. You could say “We are 95% confident that the [true ef...
(AI_HYDRA_LEO_MISSING_COMMA)
[grammar] ~132-~132: The verb ‘covariate’ does not usually follow articles like ‘a’. Check that ‘covariate’ is spelled correctly; using ‘covariate’ as a noun may be non-standard.
Context: ...using correlated pre-experiment data as a covariate. CUPED can significantly improve sen...
(A_INFINITIVE)
[uncategorized] ~134-~134: Use a comma before ‘so’ if it connects two independent clauses (unless they are closely connected and short).
Context: ...ED can significantly improve sensitivity so experiments reach conclusions faster or...
(COMMA_COMPOUND_SENTENCE_2)
[style] ~165-~165: You’ve already used the word ‘also’ once in your sentence, so using it again may be redundant.
Context: ... but also practically meaningful. It's also used in power calculations when designing ex...
(REDUNDANT_FILLER)
[style] ~179-~179: To elevate your writing, try using a synonym here.
Context: ...actions can distort results and make it hard to attribute observed effects to a sing...
(HARD_TO)
[uncategorized] ~191-~191: Possible missing comma found.
Context: ...ngful changes or underestimating effect sizes which leads to unreliable decisions. A...
(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~207-~207: Try moving the adverb to make the sentence clearer.
Context: ...irections; confirmatory experiments are designed to rigorously test a specific hypothesis. Mixing the two modes can lead to inf...
(SPLIT_INFINITIVE)
[grammar] ~226-~226: The verb ‘consider’ is used with the gerund form.
Context: ...coveries is essential. A few things to consider to reduce the risks of False Discoveries -- Star...
(ADMIT_ENJOY_VB)
[uncategorized] ~243-~243: “an” (indefinite article before a vowel sound) seems less likely than “any”.
Context: ... significant, but only 60 of those have an actual true effects, then your False Di...
(AI_HYDRA_LEO_CP_AN_ANY)
[grammar] ~246-~246: After ‘it’, use the third-person verb form “exists”.
Context: ...Failing to detect a real effect when it exist; equivalent to a Type II error. False...
(IT_VBZ)
[uncategorized] ~284-~284: Possible missing comma found.
Context: ... horizon methods are conceptually simple but are not robust to unplanned peeking or ...
(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~288-~288: ‘making a decision’ might be wordy. Consider a shorter alternative.
Context: ...an experiment for exactly two weeks and making a decision only after both weeks have completed. ...
(EN_WORDINESS_PREMIUM_MAKING_A_DECISION)
[style] ~298-~298: ‘for lack of’ might be wordy. Consider a shorter alternative.
Context: ...is, allows the experiment to stop early for lack of effect — indicating that the treatment ...
(EN_WORDINESS_PREMIUM_FOR_LACK_OF)
[uncategorized] ~316-~316: Did you mean: “By default,”?
Context: ...ces stricter control over error rates. By default GST experiment in ABsmartly uses a bind...
(BY_DEFAULT_COMMA)
[uncategorized] ~317-~317: Use a comma before ‘but’ if it connects two independent clauses (unless they are closely connected and short).
Context: ...n ABsmartly uses a binding futility type but this can be changed during the setup. ...
(COMMA_COMPOUND_SENTENCE_2)
[style] ~329-~329: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...re a few well-timed looks are enough. Group Sequential Testing is the default metho...
(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
[duplication] ~335-~335: Possible typo: you repeated a word.
Context: ...tps://absmartly.com/gst) ### Guardrail metrics Metrics monitored to ensure experiments stay wi...
(ENGLISH_WORD_REPEAT_RULE)
[uncategorized] ~359-~359: Do not mix variants of the same word (‘judgement’ and ‘judgment’) within a single text.
Context: ...ased on prior data, modelling or expert judgement. Impact estimates help prioritise whi...
(EN_WORD_COHERENCY)
[style] ~360-~360: Would you like to use the Oxford spelling “prioritize”? The spelling ‘prioritise’ is also correct.
Context: ...pert judgement. Impact estimates help prioritise which experiments to run and inform sam...
(OXFORD_SPELLING_Z_NOT_S)
[style] ~398-~398: Would you like to use the Oxford spelling “optimize”? The spelling ‘optimise’ is also correct.
Context: ...best. They are useful when you want to optimise several components together, such as he...
(OXFORD_SPELLING_Z_NOT_S)
[formatting] ~399-~399: If the ‘because’ clause is essential to the meaning, do not use a comma before the clause.
Context: ...ntly more traffic than a simple A/B test, because traffic must be spread across many vari...
(COMMA_BEFORE_BECAUSE)
[typographical] ~399-~399: Use a comma after an introductory phrase.
Context: ...tions and the analysis is more complex. For this reason Multivariate experiments are not suppor...
(COMMA_INTRODUCTORY_WORDS_PHRASES)
[style] ~403-~403: Would you like to use the Oxford spelling “optimize”? The spelling ‘optimise’ is also correct.
Context: ... ABsmartly. Example: You want to optimise a landing page with: - 2 different he...
(OXFORD_SPELLING_Z_NOT_S)
[typographical] ~403-~403: Do not use a colon (:) before a series that is introduced by a preposition (‘with’). Remove the colon or add a noun or a noun phrase after the preposition.
Context: ...* You want to optimise a landing page with: - 2 different headlines (H1, H2) - ...
(RP_COLON)
[grammar] ~406-~406: It appears that two hyphens are missing.
Context: ...ero images (I1, I2, I3) - 2 different call to action buttons (C1, C2) A multivariate expe...
(CALL_TO_ACTION_HYPHEN)
[uncategorized] ~428-~428: Possible missing preposition found.
Context: ...low.” --- ## O ### Observed effect The measured difference between treatment a...
(AI_HYDRA_LEO_MISSING_IN)
[grammar] ~430-~430: Did you mean “sample”?
Context: ...ble from the data, but it is subject to sampling variability. Example: Treatment ...
(IT_IS_JJ_TO_VBG)
[uncategorized] ~436-~436: Possible missing comma found.
Context: ... more powerful for directional questions but must be justified in advance to avoid b...
(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~442-~442: Would you like to use the Oxford spelling “utilization”? The spelling ‘utilisation’ is also correct.
Context: ... Example: Error rate, latency, CPU utilisation or cache hit rate. --- ## P ### Peek...
(OXFORD_SPELLING_Z_NOT_S)
[duplication] ~464-~464: Possible typo: you repeated a word.
Context: ... significance level. ### Pre-selection bias Bias introduced when the users who enter a s...
(ENGLISH_WORD_REPEAT_RULE)
[uncategorized] ~481-~481: Possible missing comma found.
Context: ...xperimentation is not a one-off activity but a core part of how the organisation bui...
(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~481-~481: Do not mix variants of the same word (‘organisation’ and ‘organization’) within a single text.
Context: ...off activity but a core part of how the organisation builds products. Example: Marty C...
(EN_WORD_COHERENCY)
[uncategorized] ~506-~506: Possible missing comma found.
Context: ...oring average order value and page load time while the primary metric is conversion ...
(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~516-~516: Possible missing comma found.
Context: ...* Discounts shown only to the treatment group affecting reference prices for control ...
(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~529-~529: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...effect of the planned size or larger. High power means you are less likely to miss real ...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
[uncategorized] ~530-~530: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ... likely to miss real improvements; very low power leads to many inconclusive or misleadin...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
[formatting] ~550-~550: Insert a comma before quoting reported speech: “states, “”…
Context: ...t. ### Twyman’s law A heuristic that states “the more surprising a result looks, the ...
(SAID_COMMA_SPEECH)
[uncategorized] ~583-~583: Do not mix variants of the same word (‘standardise’ and ‘standardize’) within a single text.
Context: ...me traffic. --- ## Z ### Z-score A standardised value expressing how many standard devi...
(EN_WORD_COHERENCY)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: Yarn Build
There was a problem hiding this comment.
Actionable comments posted: 4
♻️ Duplicate comments (5)
docs/glossary.mdx (5)
32-32: Fix spelling: "refered" → "referred".This was flagged in a previous review and appears to remain unresolved.
- A/B/n experiments are also sometimes refered to as multi-variant. Not to be confused with [Multivariate Experiment](#multivariate-experiment). + A/B/n experiments are also sometimes referred to as multi-variant. Not to be confused with [Multivariate Experiment](#multivariate-experiment).
92-94: Fix word repetition and add missing comma.Line 92 repeats "means", and line 94 appears to need a comma after the closing quotation mark.
- it means means that if you were to repeat the same experiment 100 times, the true effect would lie within the CI in about 95 of those. + it means that if you were to repeat the same experiment 100 times, the true effect would lie within the CI in about 95 of those. - - You could say "We are 95% confident that the [true effect](#true-effect) of the treatment is between +0.5% and +4.1%." + + You could say, "We are 95% confident that the [true effect](#true-effect) of the treatment is between +0.5% and +4.1%."
246-246: Fix subject–verb agreement: "it exist" → "it exists".This was flagged in a previous review and remains unresolved.
- Failing to detect a real effect when it exist; equivalent to a Type II error. + Failing to detect a real effect when it exists; equivalent to a Type II error.
317-318: Add missing commas and fix malformed link.Two issues here: (1) missing comma after "By default"; (2) missing comma before coordinating conjunction "but"; and (3) the link on line 333 still has quotes inside the URL parentheses.
- By default GST experiment in ABsmartly uses a binding futility type but this can be changed during the setup. + By default, GST experiment in ABsmartly uses a binding futility type, but this can be changed during the setup. - Do you want to know more about GST? Read our [dedicated GST article]('https://absmartly.com/gst') + Do you want to know more about GST? Read our [dedicated article](https://absmartly.com/gst)
422-425: Fix preposition–colon issue and add hyphens to compound modifier.Line 422 uses a colon after the preposition "with", which is grammatically incorrect. Line 425 needs "call to action" to be hyphenated as a compound modifier.
- You want to optimise a landing page with: + You want to optimise a landing page with the following: - 2 different headlines (H1, H2) - 3 different hero images (I1, I2, I3) - - 2 different call to action buttons (C1, C2) + - 2 different call-to-action buttons (C1, C2)
🧹 Nitpick comments (1)
docs/glossary.mdx (1)
602-606: Fix British–American English consistency.The file mixes British and American spellings (e.g., "randomisation" on line 18, "optimize" vs "optimise", "utilisation" vs "utilization"). Ensure consistency throughout. Given that earlier entries use British spellings ("randomisation", "analysing", "organisation"), maintain that convention.
Affected lines (partial list):
- Line 417: "optimise" (consistent with British, good)
- Line 422: "optimise" (good)
- Line 604: "standardised" (good)
Consider flagging any remaining American spellings in copy-editing pass, though this is a lower priority.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
docs/glossary.mdx(1 hunks)
🧰 Additional context used
🪛 LanguageTool
docs/glossary.mdx
[grammar] ~15-~15: If ‘type’ is a classification term, ‘A’ is not necessary. Use “type of”. (The phrases ‘kind of’ and ‘sort of’ are informal if they mean ‘to some extent’.)
Context: ...- ## A ### A/A experiment A special type of A/B experiment where users are randomly s...
(KIND_OF_A)
[style] ~18-~18: Would you like to use the Oxford spelling “randomization”? The spelling ‘randomisation’ is also correct.
Context: ...M), tracking bugs or unexpected bias in randomisation before you start testing real changes. ...
(OXFORD_SPELLING_Z_NOT_S)
[grammar] ~31-~31: If ‘type’ is a classification term, ‘A’ is not necessary. Use “type of”. (The phrases ‘kind of’ and ‘sort of’ are informal if they mean ‘to some extent’.)
Context: ...rate. ### A/B/n experiment A special type of A/B experiment that compares a control to...
(KIND_OF_A)
[style] ~41-~41: Would you like to use the Oxford spelling “generalizability”? The spelling ‘generalisability’ is also correct.
Context: ...eting can reduce sample size and affect generalisability. Example: Running an experiment on...
(OXFORD_SPELLING_Z_NOT_S)
[duplication] ~92-~92: Possible typo: you repeated a word.
Context: ...fidence interval of [+0.5%, +4.1%], it means means that if you were to repeat the same exp...
(ENGLISH_WORD_REPEAT_RULE)
[uncategorized] ~94-~94: Possible missing comma found.
Context: ...the CI in about 95 of those. You could say “We are 95% confident that the [true ef...
(AI_HYDRA_LEO_MISSING_COMMA)
[grammar] ~132-~132: The verb ‘covariate’ does not usually follow articles like ‘a’. Check that ‘covariate’ is spelled correctly; using ‘covariate’ as a noun may be non-standard.
Context: ...using correlated pre-experiment data as a covariate. CUPED can significantly improve sen...
(A_INFINITIVE)
[uncategorized] ~134-~134: Use a comma before ‘so’ if it connects two independent clauses (unless they are closely connected and short).
Context: ...ED can significantly improve sensitivity so experiments reach conclusions faster or...
(COMMA_COMPOUND_SENTENCE_2)
[style] ~165-~165: You’ve already used the word ‘also’ once in your sentence, so using it again may be redundant.
Context: ... but also practically meaningful. It's also used in power calculations when designing ex...
(REDUNDANT_FILLER)
[style] ~179-~179: To elevate your writing, try using a synonym here.
Context: ...actions can distort results and make it hard to attribute observed effects to a sing...
(HARD_TO)
[uncategorized] ~191-~191: Possible missing comma found.
Context: ...ngful changes or underestimating effect sizes which leads to unreliable decisions. A...
(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~207-~207: Try moving the adverb to make the sentence clearer.
Context: ...irections; confirmatory experiments are designed to rigorously test a specific hypothesis. Mixing the two modes can lead to inf...
(SPLIT_INFINITIVE)
[grammar] ~226-~226: The verb ‘consider’ is used with the gerund form.
Context: ...coveries is essential. A few things to consider to reduce the risks of False Discoveries -ehavio...
(ADMIT_ENJOY_VB)
[uncategorized] ~243-~243: “an” (indefinite article before a vowel sound) seems less likely than “any”.
Context: ... significant, but only 60 of those have an actual true effects, then your False Di...
(AI_HYDRA_LEO_CP_AN_ANY)
[grammar] ~246-~246: After ‘it’, use the third-person verb form “exists”.
Context: ...Failing to detect a real effect when it exist; equivalent to a Type II error. False...
(IT_VBZ)
[uncategorized] ~284-~284: Possible missing comma found.
Context: ... horizon methods are conceptually simple but are not robust to unplanned peeking or ...
(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~288-~288: ‘making a decision’ might be wordy. Consider a shorter alternative.
Context: ...an experiment for exactly two weeks and making a decision only after both weeks have completed. ...
(EN_WORDINESS_PREMIUM_MAKING_A_DECISION)
[style] ~298-~298: ‘for lack of’ might be wordy. Consider a shorter alternative.
Context: ...is, allows the experiment to stop early for lack of effect — indicating that the treatment ...
(EN_WORDINESS_PREMIUM_FOR_LACK_OF)
[uncategorized] ~316-~316: Did you mean: “By default,”?
Context: ...ces stricter control over error rates. By default GST experiment in ABsmartly uses a bind...
(BY_DEFAULT_COMMA)
[uncategorized] ~317-~317: Use a comma before ‘but’ if it connects two independent clauses (unless they are closely connected and short).
Context: ...n ABsmartly uses a binding futility type but this can be changed during the setup. ...
(COMMA_COMPOUND_SENTENCE_2)
[style] ~329-~329: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...re a few well-timed looks are enough. Group Sequential Testing is the default metho...
(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
[duplication] ~335-~335: Possible typo: you repeated a word.
Context: ...tps://absmartly.com/gst) ### Guardrail metrics Metrics monitored to ensure experiments stay wi...
(ENGLISH_WORD_REPEAT_RULE)
[grammar] ~394-~394: This phrase is duplicated. You should probably use “Metric variance” only once.
Context: ...users gives a mean of 50 per user. ### Metric variance Metric variance refers to the amount of variability or ...
(PHRASE_REPETITION)
[style] ~417-~417: Would you like to use the Oxford spelling “optimize”? The spelling ‘optimise’ is also correct.
Context: ...best. They are useful when you want to optimise several components together, such as he...
(OXFORD_SPELLING_Z_NOT_S)
[formatting] ~418-~418: If the ‘because’ clause is essential to the meaning, do not use a comma before the clause.
Context: ...ntly more traffic than a simple A/B test, because traffic must be spread across many vari...
(COMMA_BEFORE_BECAUSE)
[typographical] ~418-~418: Use a comma after an introductory phrase.
Context: ...tions and the analysis is more complex. For this reason Multivariate experiments are not suppor...
(COMMA_INTRODUCTORY_WORDS_PHRASES)
[style] ~422-~422: Would you like to use the Oxford spelling “optimize”? The spelling ‘optimise’ is also correct.
Context: ... ABsmartly. Example: You want to optimise a landing page with: - 2 different he...
(OXFORD_SPELLING_Z_NOT_S)
[typographical] ~422-~422: Do not use a colon (:) before a series that is introduced by a preposition (‘with’). Remove the colon or add a noun or a noun phrase after the preposition.
Context: ...* You want to optimise a landing page with: - 2 different headlines (H1, H2) - ...
(RP_COLON)
[grammar] ~425-~425: It appears that two hyphens are missing.
Context: ...ero images (I1, I2, I3) - 2 different call to action buttons (C1, C2) A multivariate expe...
(CALL_TO_ACTION_HYPHEN)
[uncategorized] ~447-~447: Possible missing preposition found.
Context: ...low.” --- ## O ### Observed effect The measured difference between treatment a...
(AI_HYDRA_LEO_MISSING_IN)
[grammar] ~449-~449: Did you mean “sample”?
Context: ...ble from the data, but it is subject to sampling variability. Example: Treatment ...
(IT_IS_JJ_TO_VBG)
[uncategorized] ~455-~455: Possible missing comma found.
Context: ... more powerful for directional questions but must be justified in advance to avoid b...
(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~461-~461: Would you like to use the Oxford spelling “utilization”? The spelling ‘utilisation’ is also correct.
Context: ... Example: Error rate, latency, CPU utilisation or cache hit rate. ### Outliers --- ...
(OXFORD_SPELLING_Z_NOT_S)
[duplication] ~485-~485: Possible typo: you repeated a word.
Context: ... significance level. ### Pre-selection bias Bias introduced when the users who enter a s...
(ENGLISH_WORD_REPEAT_RULE)
[uncategorized] ~502-~502: Possible missing comma found.
Context: ...xperimentation is not a one-off activity but a core part of how the organisation bui...
(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~502-~502: Do not mix variants of the same word (‘organisation’ and ‘organization’) within a single text.
Context: ...off activity but a core part of how the organisation builds products. Example: Marty C...
(EN_WORD_COHERENCY)
[uncategorized] ~527-~527: Possible missing comma found.
Context: ...oring average order value and page load time while the primary metric is conversion ...
(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~537-~537: Possible missing comma found.
Context: ...* Discounts shown only to the treatment group affecting reference prices for control ...
(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~550-~550: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...effect of the planned size or larger. High power means you are less likely to miss real ...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
[uncategorized] ~551-~551: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ... likely to miss real improvements; very low power leads to many inconclusive or misleadin...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
[formatting] ~571-~571: Insert a comma before quoting reported speech: “states, “”…
Context: ...t. ### Twyman’s law A heuristic that states “the more surprising a result looks, the ...
(SAID_COMMA_SPEECH)
[uncategorized] ~604-~604: Do not mix variants of the same word (‘standardise’ and ‘standardize’) within a single text.
Context: ...me traffic. --- ## Z ### Z-score A standardised value expressing how many standard devi...
(EN_WORD_COHERENCY)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: Yarn Build
docs/glossary.mdx
Outdated
| ### Point estimate | ||
|
|
||
| ### Power level | ||
|
|
||
| See [Experimentation power](#experimentation-power). |
There was a problem hiding this comment.
Complete the "Point estimate" and "Power level" definitions.
Both entries (lines 474–478) are incomplete or point to other sections without their own content.
### Point estimate
-
+The single best numerical estimate of a parameter (such as effect size) computed from the observed data.
+
+Point estimates do not convey uncertainty; they are always accompanied by confidence intervals or standard errors.
+
+**Example:** An observed lift of +3.2% is the point estimate; the 95% CI of [+0.1%, +6.3%] conveys uncertainty.
+Committable suggestion skipped: line range outside the PR's diff.
🤖 Prompt for AI Agents
In docs/glossary.mdx around lines 474 to 478, the "Point estimate" and "Power
level" glossary entries are empty or merely reference other sections; add
concise definitions for both: for "Point estimate" define it as a single-value
statistic (e.g., sample mean or proportion) used to estimate a population
parameter and optionally mention typical uses and contrast with interval
estimates; for "Power level" provide a short definition (statistical power)
describing the probability of correctly rejecting a false null hypothesis, note
typical target values (e.g., 80% or 90%), and keep "See [Experimentation power]"
as a cross-reference for details. Ensure wording matches glossary tone and
length of neighboring entries.
There was a problem hiding this comment.
Actionable comments posted: 0
♻️ Duplicate comments (9)
docs/glossary.mdx (9)
92-92: Remove duplicated word "means".Line 92 repeats the word "means" and should read as a single instance.
Apply this diff:
-it means means that if you were to repeat the same experiment 100 times, the true effect would lie within the CI in about 95 of those. +it means that if you were to repeat the same experiment 100 times, the true effect would lie within the CI in about 95 of those.
228-228: Fix malformed text in False discovery section.Line 228 starts with "-ehavior" which appears to be a truncated word; it should begin "Ground hypotheses in user research, behaviour...".
Apply this diff:
--ehavior, prior data, or product theory. Avoid "spaghetti testing" — randomly trying ideas just to see what sticks. +- Ground hypotheses in user research, behaviour, prior data, or product theory. Avoid "spaghetti testing" — randomly trying ideas just to see what sticks.
626-627: Complete True effect definition.Lines 626–627 show only a heading with no definition or example. Add content matching the structure of neighbouring entries.
Apply this diff:
### True effect - +The actual, population-level effect of the treatment—what would be observed with infinite data and no noise. + +In practice, we never know the true effect with certainty; experiments estimate it via point estimates and confidence intervals. + +**Example:** The true effect of the new button design is +2.1% conversion lift (unknown until observed across infinite users).
316-317: Add missing commas in GST futility type definition.Lines 316–317 require commas for correct punctuation: one after the introductory phrase "By default" and one before the coordinating conjunction "but".
Apply this diff:
-By default GST experiment in ABsmartly uses a binding futility type but this can be changed during the setup. +By default, GST experiment in ABsmartly uses a binding futility type, but this can be changed during the setup.
444-444: Remove colon after preposition or restructure sentence.Line 444 incorrectly places a colon after the preposition "with". Either remove the colon or add a noun phrase following the preposition.
Apply this diff:
-You want to optimise a landing page with: +You want to optimise a landing page with the following:Or alternatively:
-You want to optimise a landing page with: +You want to optimise a landing page that includes:
363-363: Hyphenate compound modifier "call-to-action".Line 363 (shown at line 447 in output) should use hyphens when the phrase modifies a noun directly.
Apply this diff:
-- 2 different call to action buttons (C1, C2) +- 2 different call-to-action buttons (C1, C2)
246-246: Fix subject–verb agreement: "exist" → "exists".Line 246 uses incorrect verb form after "it"; third-person singular requires "exists".
Apply this diff:
-Failing to detect a real effect when it exist; equivalent to a Type II error. +Failing to detect a real effect when it exists; equivalent to a Type II error.
32-32: Fix spelling error: "refered" → "referred".This entry contains a misspelling that appears to persist from previous reviews marked as addressed.
Apply this diff:
-A/B/n experiments are also sometimes refered to as multi-variant. Not to be confused with [Multivariate Experiment](#multivariate-experiment). +A/B/n experiments are also sometimes referred to as multi-variant. Not to be confused with [Multivariate Experiment](#multivariate-experiment).
537-541: Complete empty Point estimate and Power level definitions.Lines 537–541 show only headings with no definitions or content. Both entries should provide concise explanations matching neighbouring glossary entries.
For "Point estimate", add:
### Point estimate - +The single best numerical estimate of a parameter (such as effect size) computed from observed data. + +Point estimates do not convey uncertainty; they are always accompanied by confidence intervals or standard errors. + +**Example:** An observed lift of +3.2% is the point estimate; the 95% CI of [+0.1%, +6.3%] conveys uncertainty.For "Power level", update the reference:
### Power level - +The probability of correctly detecting a true effect if one exists; see [Experimentation power](#experimentation-power) for more detail.
🧹 Nitpick comments (1)
docs/glossary.mdx (1)
667-667: Standardize spelling variant: "standardised" vs "standardized".Line 667 uses British spelling "standardised", while other entries mix both "optimise/optimize" variants. Ensure consistency throughout the glossary or adopt a single convention.
Suggested: apply British English throughout (standardised, optimise, etc.) or American English throughout.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
docs/glossary.mdx(1 hunks)
🧰 Additional context used
🪛 LanguageTool
docs/glossary.mdx
[grammar] ~15-~15: If ‘type’ is a classification term, ‘A’ is not necessary. Use “type of”. (The phrases ‘kind of’ and ‘sort of’ are informal if they mean ‘to some extent’.)
Context: ...- ## A ### A/A experiment A special type of A/B experiment where users are randomly s...
(KIND_OF_A)
[style] ~18-~18: Would you like to use the Oxford spelling “randomization”? The spelling ‘randomisation’ is also correct.
Context: ...M), tracking bugs or unexpected bias in randomisation before you start testing real changes. ...
(OXFORD_SPELLING_Z_NOT_S)
[grammar] ~31-~31: If ‘type’ is a classification term, ‘A’ is not necessary. Use “type of”. (The phrases ‘kind of’ and ‘sort of’ are informal if they mean ‘to some extent’.)
Context: ...rate. ### A/B/n experiment A special type of A/B experiment that compares a control to...
(KIND_OF_A)
[style] ~41-~41: Would you like to use the Oxford spelling “generalizability”? The spelling ‘generalisability’ is also correct.
Context: ...eting can reduce sample size and affect generalisability. Example: Running an experiment on...
(OXFORD_SPELLING_Z_NOT_S)
[duplication] ~92-~92: Possible typo: you repeated a word.
Context: ...fidence interval of [+0.5%, +4.1%], it means means that if you were to repeat the same exp...
(ENGLISH_WORD_REPEAT_RULE)
[uncategorized] ~94-~94: Possible missing comma found.
Context: ...the CI in about 95 of those. You could say “We are 95% confident that the [true ef...
(AI_HYDRA_LEO_MISSING_COMMA)
[grammar] ~132-~132: The verb ‘covariate’ does not usually follow articles like ‘a’. Check that ‘covariate’ is spelled correctly; using ‘covariate’ as a noun may be non-standard.
Context: ...using correlated pre-experiment data as a covariate. CUPED can significantly improve sen...
(A_INFINITIVE)
[uncategorized] ~134-~134: Use a comma before ‘so’ if it connects two independent clauses (unless they are closely connected and short).
Context: ...ED can significantly improve sensitivity so experiments reach conclusions faster or...
(COMMA_COMPOUND_SENTENCE_2)
[style] ~165-~165: You’ve already used the word ‘also’ once in your sentence, so using it again may be redundant.
Context: ... but also practically meaningful. It's also used in power calculations when designing ex...
(REDUNDANT_FILLER)
[style] ~179-~179: To elevate your writing, try using a synonym here.
Context: ...actions can distort results and make it hard to attribute observed effects to a sing...
(HARD_TO)
[uncategorized] ~191-~191: Possible missing comma found.
Context: ...ngful changes or underestimating effect sizes which leads to unreliable decisions. A...
(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~207-~207: Try moving the adverb to make the sentence clearer.
Context: ...irections; confirmatory experiments are designed to rigorously test a specific hypothesis. Mixing the two modes can lead to inf...
(SPLIT_INFINITIVE)
[grammar] ~226-~226: The verb ‘consider’ is used with the gerund form.
Context: ...coveries is essential. A few things to consider to reduce the risks of False Discoveries -ehavio...
(ADMIT_ENJOY_VB)
[uncategorized] ~243-~243: “an” (indefinite article before a vowel sound) seems less likely than “any”.
Context: ... significant, but only 60 of those have an actual true effects, then your False Di...
(AI_HYDRA_LEO_CP_AN_ANY)
[grammar] ~246-~246: After ‘it’, use the third-person verb form “exists”.
Context: ...Failing to detect a real effect when it exist; equivalent to a Type II error. False...
(IT_VBZ)
[uncategorized] ~284-~284: Possible missing comma found.
Context: ... horizon methods are conceptually simple but are not robust to unplanned peeking or ...
(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~288-~288: ‘making a decision’ might be wordy. Consider a shorter alternative.
Context: ...an experiment for exactly two weeks and making a decision only after both weeks have completed. ...
(EN_WORDINESS_PREMIUM_MAKING_A_DECISION)
[style] ~298-~298: ‘for lack of’ might be wordy. Consider a shorter alternative.
Context: ...is, allows the experiment to stop early for lack of effect — indicating that the treatment ...
(EN_WORDINESS_PREMIUM_FOR_LACK_OF)
[uncategorized] ~316-~316: Did you mean: “By default,”?
Context: ...ces stricter control over error rates. By default GST experiment in ABsmartly uses a bind...
(BY_DEFAULT_COMMA)
[uncategorized] ~317-~317: Use a comma before ‘but’ if it connects two independent clauses (unless they are closely connected and short).
Context: ...n ABsmartly uses a binding futility type but this can be changed during the setup. ...
(COMMA_COMPOUND_SENTENCE_2)
[style] ~329-~329: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...re a few well-timed looks are enough. Group Sequential Testing is the default metho...
(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
[duplication] ~335-~335: Possible typo: you repeated a word.
Context: ...tps://absmartly.com/gst) ### Guardrail metrics Metrics monitored to ensure experiments stay wi...
(ENGLISH_WORD_REPEAT_RULE)
[uncategorized] ~357-~357: Possible missing comma found.
Context: ...able prediction about the outcome of an experiment usually describing how a change (treatm...
(AI_HYDRA_LEO_MISSING_COMMA)
[grammar] ~416-~416: This phrase is duplicated. You should probably use “Metric variance” only once.
Context: ...users gives a mean of 50 per user. ### Metric variance Metric variance refers to the amount of variability or ...
(PHRASE_REPETITION)
[style] ~439-~439: Would you like to use the Oxford spelling “optimize”? The spelling ‘optimise’ is also correct.
Context: ...best. They are useful when you want to optimise several components together, such as he...
(OXFORD_SPELLING_Z_NOT_S)
[formatting] ~440-~440: If the ‘because’ clause is essential to the meaning, do not use a comma before the clause.
Context: ...ntly more traffic than a simple A/B test, because traffic must be spread across many vari...
(COMMA_BEFORE_BECAUSE)
[typographical] ~440-~440: Use a comma after an introductory phrase.
Context: ...tions and the analysis is more complex. For this reason Multivariate experiments are not suppor...
(COMMA_INTRODUCTORY_WORDS_PHRASES)
[style] ~444-~444: Would you like to use the Oxford spelling “optimize”? The spelling ‘optimise’ is also correct.
Context: ... ABsmartly. Example: You want to optimise a landing page with: - 2 different he...
(OXFORD_SPELLING_Z_NOT_S)
[typographical] ~444-~444: Do not use a colon (:) before a series that is introduced by a preposition (‘with’). Remove the colon or add a noun or a noun phrase after the preposition.
Context: ...* You want to optimise a landing page with: - 2 different headlines (H1, H2) - ...
(RP_COLON)
[grammar] ~447-~447: It appears that two hyphens are missing.
Context: ...ero images (I1, I2, I3) - 2 different call to action buttons (C1, C2) A multivariate expe...
(CALL_TO_ACTION_HYPHEN)
[style] ~462-~462: If ‘chance’ means ‘possibility’, this phrase is redundant. Consider writing “chance”.
Context: ... that any observed difference is due to random chance. The null hypothesis is the foundation...
(RANDOM_CHANCE)
[uncategorized] ~485-~485: Possible missing preposition found.
Context: ...ence. --- ## O ### Observed effect The measured difference between treatment a...
(AI_HYDRA_LEO_MISSING_IN)
[grammar] ~487-~487: Did you mean “sample”?
Context: ...ble from the data, but it is subject to sampling variability. Example: Treatment ...
(IT_IS_JJ_TO_VBG)
[uncategorized] ~495-~495: Possible missing comma found.
Context: ...detect effects with smaller sample sizes but only when you care about a change in on...
(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~512-~512: Would you like to use the Oxford spelling “utilization”? The spelling ‘utilisation’ is also correct.
Context: ... Example: Error rate, latency, CPU utilisation or cache hit rate. ### Outliers Outlie...
(OXFORD_SPELLING_Z_NOT_S)
[duplication] ~514-~514: Possible typo: you repeated a word.
Context: ...CPU utilisation or cache hit rate. ### Outliers Outliers are data points that are significantly ...
(ENGLISH_WORD_REPEAT_RULE)
[duplication] ~548-~548: Possible typo: you repeated a word.
Context: ... significance level. ### Pre-selection bias Bias introduced when the users who enter a s...
(ENGLISH_WORD_REPEAT_RULE)
[uncategorized] ~565-~565: Possible missing comma found.
Context: ...xperimentation is not a one-off activity but a core part of how the organisation bui...
(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~565-~565: Do not mix variants of the same word (‘organisation’ and ‘organization’) within a single text.
Context: ...off activity but a core part of how the organisation builds products. Example: Marty C...
(EN_WORD_COHERENCY)
[uncategorized] ~590-~590: Possible missing comma found.
Context: ...oring average order value and page load time while the primary metric is conversion ...
(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~600-~600: Possible missing comma found.
Context: ...* Discounts shown only to the treatment group affecting reference prices for control ...
(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~613-~613: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...effect of the planned size or larger. High power means you are less likely to miss real ...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
[uncategorized] ~614-~614: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ... likely to miss real improvements; very low power leads to many inconclusive or misleadin...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
[formatting] ~634-~634: Insert a comma before quoting reported speech: “states, “”…
Context: ...t. ### Twyman’s law A heuristic that states “the more surprising a result looks, the ...
(SAID_COMMA_SPEECH)
[uncategorized] ~667-~667: Do not mix variants of the same word (‘standardise’ and ‘standardize’) within a single text.
Context: ...me traffic. --- ## Z ### Z-score A standardised value expressing how many standard devi...
(EN_WORD_COHERENCY)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: Redirect rules - absmartly-docs
- GitHub Check: Header rules - absmartly-docs
- GitHub Check: Pages changed - absmartly-docs
- GitHub Check: Yarn Build
🔇 Additional comments (1)
docs/glossary.mdx (1)
514-522: Approve Outliers definition.The "Outliers" entry now includes a complete definition explaining extreme values, their impact on variance, and a practical example, which addresses the previous gap.
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (7)
docs/glossary.mdx (7)
30-36: Fix spelling in A/B/n entry (“refered” → “referred”).- A/B/n experiments are also sometimes refered to as multi-variant. Not to be confused with [Multivariate Experiment](#multivariate-experiment). + A/B/n experiments are also sometimes referred to as multi-variant. Not to be confused with [Multivariate Experiment](#multivariate-experiment).
91-93: Remove duplicated word (“it means means”).- it means means that if you were to repeat the same experiment 100 times, the true effect would lie within the CI in about 95 of those. + it means that if you were to repeat the same experiment 100 times, the true effect would lie within the CI in about 95 of those.
216-234: Fix malformed bullet in “False discovery” section (currently renders as broken text).Line 228 looks truncated (“-ehavior, …”) and should be a proper bullet.
- A few things to consider to reduce the risks of False Discoveries - --ehavior, prior data, or product theory. Avoid “spaghetti testing” — randomly trying ideas just to see what sticks. + A few things to consider to reduce the risks of false discoveries: + - Ground hypotheses in user research, behaviour, prior data, or product theory. Avoid “spaghetti testing” — randomly trying ideas just to see what sticks.
245-249: Fix subject–verb agreement (“when it exist” → “when it exists”).- Failing to detect a real effect when it exist; equivalent to a Type II error. + Failing to detect a real effect when it exists; equivalent to a Type II error.
317-334: Tighten GST paragraph punctuation/grammar and fix “80% than” sentence.Suggested edits:
- By default GST experiment in ABsmartly uses a binding futility type but this can be changed during the setup. + By default, GST experiments in ABsmartly use a binding futility type, but this can be changed during setup.- Group Sequential Testing is the default method when creating new experiment. - It leads to making decisions up to 80% than with a more traditional [Fixed Horizon Experiment](#fixed-horizon-testing). + Group Sequential Testing is the default method when creating a new experiment. + It can lead to decisions up to 80% faster than a more traditional [Fixed horizon testing](#fixed-horizon-testing).
443-448: Fix “with:” colon usage and hyphenate “call-to-action”.- You want to optimise a landing page with: + You want to optimise a landing page with the following: - 2 different headlines (H1, H2) - 3 different hero images (I1, I2, I3) - - 2 different call to action buttons (C1, C2) + - 2 different call-to-action buttons (C1, C2)
637-639: “True effect” entry is empty—add a definition or remove the heading.Right now it’s just a heading; consider adding a short definition + example (consistent with other entries).
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
docs/glossary.mdx(1 hunks)
🧰 Additional context used
🪛 LanguageTool
docs/glossary.mdx
[grammar] ~15-~15: If ‘type’ is a classification term, ‘A’ is not necessary. Use “type of”. (The phrases ‘kind of’ and ‘sort of’ are informal if they mean ‘to some extent’.)
Context: ...- ## A ### A/A experiment A special type of A/B experiment where users are randomly s...
(KIND_OF_A)
[style] ~18-~18: Would you like to use the Oxford spelling “randomization”? The spelling ‘randomisation’ is also correct.
Context: ...M), tracking bugs or unexpected bias in randomisation before you start testing real changes. ...
(OXFORD_SPELLING_Z_NOT_S)
[grammar] ~31-~31: If ‘type’ is a classification term, ‘A’ is not necessary. Use “type of”. (The phrases ‘kind of’ and ‘sort of’ are informal if they mean ‘to some extent’.)
Context: ...rate. ### A/B/n experiment A special type of A/B experiment that compares a control to...
(KIND_OF_A)
[style] ~41-~41: Would you like to use the Oxford spelling “generalizability”? The spelling ‘generalisability’ is also correct.
Context: ...eting can reduce sample size and affect generalisability. Example: Running an experiment on...
(OXFORD_SPELLING_Z_NOT_S)
[duplication] ~92-~92: Possible typo: you repeated a word.
Context: ...fidence interval of [+0.5%, +4.1%], it means means that if you were to repeat the same exp...
(ENGLISH_WORD_REPEAT_RULE)
[uncategorized] ~94-~94: Possible missing comma found.
Context: ...the CI in about 95 of those. You could say “We are 95% confident that the [true ef...
(AI_HYDRA_LEO_MISSING_COMMA)
[grammar] ~132-~132: The verb ‘covariate’ does not usually follow articles like ‘a’. Check that ‘covariate’ is spelled correctly; using ‘covariate’ as a noun may be non-standard.
Context: ...using correlated pre-experiment data as a covariate. CUPED can significantly improve sen...
(A_INFINITIVE)
[uncategorized] ~134-~134: Use a comma before ‘so’ if it connects two independent clauses (unless they are closely connected and short).
Context: ...ED can significantly improve sensitivity so experiments reach conclusions faster or...
(COMMA_COMPOUND_SENTENCE_2)
[style] ~165-~165: You’ve already used the word ‘also’ once in your sentence, so using it again may be redundant.
Context: ... but also practically meaningful. It's also used in power calculations when designing ex...
(REDUNDANT_FILLER)
[style] ~179-~179: To elevate your writing, try using a synonym here.
Context: ...actions can distort results and make it hard to attribute observed effects to a sing...
(HARD_TO)
[uncategorized] ~191-~191: Possible missing comma found.
Context: ...ngful changes or underestimating effect sizes which leads to unreliable decisions. A...
(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~207-~207: Try moving the adverb to make the sentence clearer.
Context: ...irections; confirmatory experiments are designed to rigorously test a specific hypothesis. Mixing the two modes can lead to inf...
(SPLIT_INFINITIVE)
[grammar] ~226-~226: The verb ‘consider’ is used with the gerund form.
Context: ...coveries is essential. A few things to consider to reduce the risks of False Discoveries -ehavio...
(ADMIT_ENJOY_VB)
[uncategorized] ~243-~243: “an” (indefinite article before a vowel sound) seems less likely than “any”.
Context: ... significant, but only 60 of those have an actual true effects, then your False Di...
(AI_HYDRA_LEO_CP_AN_ANY)
[grammar] ~246-~246: After ‘it’, use the third-person verb form “exists”.
Context: ...Failing to detect a real effect when it exist; equivalent to a Type II error. False...
(IT_VBZ)
[uncategorized] ~284-~284: Possible missing comma found.
Context: ... horizon methods are conceptually simple but are not robust to unplanned peeking or ...
(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~288-~288: ‘making a decision’ might be wordy. Consider a shorter alternative.
Context: ...an experiment for exactly two weeks and making a decision only after both weeks have completed. ...
(EN_WORDINESS_PREMIUM_MAKING_A_DECISION)
[style] ~298-~298: ‘for lack of’ might be wordy. Consider a shorter alternative.
Context: ...is, allows the experiment to stop early for lack of effect — indicating that the treatment ...
(EN_WORDINESS_PREMIUM_FOR_LACK_OF)
[uncategorized] ~316-~316: Did you mean: “By default,”?
Context: ...ces stricter control over error rates. By default GST experiment in ABsmartly uses a bind...
(BY_DEFAULT_COMMA)
[uncategorized] ~317-~317: Use a comma before ‘but’ if it connects two independent clauses (unless they are closely connected and short).
Context: ...n ABsmartly uses a binding futility type but this can be changed during the setup. ...
(COMMA_COMPOUND_SENTENCE_2)
[style] ~329-~329: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...re a few well-timed looks are enough. Group Sequential Testing is the default metho...
(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
[duplication] ~335-~335: Possible typo: you repeated a word.
Context: ...tps://absmartly.com/gst) ### Guardrail metrics Metrics monitored to ensure experiments stay wi...
(ENGLISH_WORD_REPEAT_RULE)
[uncategorized] ~357-~357: Possible missing comma found.
Context: ...able prediction about the outcome of an experiment usually describing how a change (treatm...
(AI_HYDRA_LEO_MISSING_COMMA)
[grammar] ~416-~416: This phrase is duplicated. You should probably use “Metric variance” only once.
Context: ...users gives a mean of 50 per user. ### Metric variance Metric variance refers to the amount of variability or ...
(PHRASE_REPETITION)
[style] ~439-~439: Would you like to use the Oxford spelling “optimize”? The spelling ‘optimise’ is also correct.
Context: ...best. They are useful when you want to optimise several components together, such as he...
(OXFORD_SPELLING_Z_NOT_S)
[formatting] ~440-~440: If the ‘because’ clause is essential to the meaning, do not use a comma before the clause.
Context: ...ntly more traffic than a simple A/B test, because traffic must be spread across many vari...
(COMMA_BEFORE_BECAUSE)
[typographical] ~440-~440: Use a comma after an introductory phrase.
Context: ...tions and the analysis is more complex. For this reason Multivariate experiments are not suppor...
(COMMA_INTRODUCTORY_WORDS_PHRASES)
[style] ~444-~444: Would you like to use the Oxford spelling “optimize”? The spelling ‘optimise’ is also correct.
Context: ... ABsmartly. Example: You want to optimise a landing page with: - 2 different he...
(OXFORD_SPELLING_Z_NOT_S)
[typographical] ~444-~444: Do not use a colon (:) before a series that is introduced by a preposition (‘with’). Remove the colon or add a noun or a noun phrase after the preposition.
Context: ...* You want to optimise a landing page with: - 2 different headlines (H1, H2) - ...
(RP_COLON)
[grammar] ~447-~447: It appears that two hyphens are missing.
Context: ...ero images (I1, I2, I3) - 2 different call to action buttons (C1, C2) A multivariate expe...
(CALL_TO_ACTION_HYPHEN)
[style] ~462-~462: If ‘chance’ means ‘possibility’, this phrase is redundant. Consider writing “chance”.
Context: ... that any observed difference is due to random chance. The null hypothesis is the foundation...
(RANDOM_CHANCE)
[uncategorized] ~485-~485: Possible missing preposition found.
Context: ...ence. --- ## O ### Observed effect The measured difference between treatment a...
(AI_HYDRA_LEO_MISSING_IN)
[grammar] ~487-~487: Did you mean “sample”?
Context: ...ble from the data, but it is subject to sampling variability. Example: Treatment ...
(IT_IS_JJ_TO_VBG)
[uncategorized] ~495-~495: Possible missing comma found.
Context: ...detect effects with smaller sample sizes but only when you care about a change in on...
(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~512-~512: Would you like to use the Oxford spelling “utilization”? The spelling ‘utilisation’ is also correct.
Context: ... Example: Error rate, latency, CPU utilisation or cache hit rate. ### Outliers Outlie...
(OXFORD_SPELLING_Z_NOT_S)
[duplication] ~514-~514: Possible typo: you repeated a word.
Context: ...CPU utilisation or cache hit rate. ### Outliers Outliers are data points that are significantly ...
(ENGLISH_WORD_REPEAT_RULE)
[uncategorized] ~546-~546: Possible missing comma found.
Context: ... experiment is a requirement for making good reliable data-informed decisions. ABsm...
(AI_HYDRA_LEO_MISSING_COMMA)
[duplication] ~552-~552: Possible typo: you repeated a word.
Context: ... significance level. ### Pre-selection bias Bias introduced when the users who enter a s...
(ENGLISH_WORD_REPEAT_RULE)
[uncategorized] ~562-~562: The noun “decision-making” (= the process of deciding something) is spelled with a hyphen.
Context: ...e primary metric. This will be the main decision making metric. It is usually good practice to ...
(DECISION_MAKING)
[uncategorized] ~575-~575: Possible missing comma found.
Context: ...xperimentation is not a one-off activity but a core part of how the organisation bui...
(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~575-~575: Do not mix variants of the same word (‘organisation’ and ‘organization’) within a single text.
Context: ...off activity but a core part of how the organisation builds products. ### P-value The p...
(EN_WORD_COHERENCY)
[uncategorized] ~601-~601: Possible missing comma found.
Context: ...oring average order value and page load time while the primary metric is conversion ...
(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~611-~611: Possible missing comma found.
Context: ...* Discounts shown only to the treatment group affecting reference prices for control ...
(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~624-~624: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...effect of the planned size or larger. High power means you are less likely to miss real ...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
[uncategorized] ~625-~625: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ... likely to miss real improvements; very low power leads to many inconclusive or misleadin...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
[formatting] ~645-~645: Insert a comma before quoting reported speech: “states, “”…
Context: ...t. ### Twyman’s law A heuristic that states “the more surprising a result looks, the ...
(SAID_COMMA_SPEECH)
[uncategorized] ~679-~679: Do not mix variants of the same word (‘standardise’ and ‘standardize’) within a single text.
Context: ...me traffic. --- ## Z ### Z-score A standardised value expressing how many standard devi...
(EN_WORD_COHERENCY)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: Yarn Build
There was a problem hiding this comment.
Actionable comments posted: 3
♻️ Duplicate comments (9)
docs/glossary.mdx (9)
251-251: Fix subject–verb agreement.Line 251 uses "when it exist" but requires the third-person verb form "exists".
🔎 Proposed fix
-Failing to detect a real effect when it exist; equivalent to a Type II error. +Failing to detect a real effect when it exists; equivalent to a Type II error.
571-571: Fix spelling error.Line 571 contains "deign" which should be "design".
🔎 Proposed fix
-ABsmartly built-in power calculator makes it easy to deign your experiment correctly. +ABsmartly's built-in power calculator makes it easy to design your experiment correctly.Note: Also added apostrophe to "ABsmartly's" for correct possessive form.
97-97: Remove duplicated word.Line 97 contains "it means means" with a duplicated word.
🔎 Proposed fix
-it means means that if you were to repeat the same experiment 100 times, the true effect would lie within the CI in about 95 of those. +it means that if you were to repeat the same experiment 100 times, the true effect would lie within the CI in about 95 of those.
321-322: Add missing commas for clarity.Lines 321-322 are missing two commas: one after the introductory phrase "By default" and one before the coordinating conjunction "but".
🔎 Proposed fix
-By default GST experiment in ABsmartly uses a binding futility type but this can be changed during the setup. +By default, GST experiment in ABsmartly uses a binding futility type, but this can be changed during the setup.
525-525: Fix subject-verb agreement and grammar.Line 525 uses singular "experiment" where plural "experiments" is needed, and the phrasing could be more concise.
🔎 Proposed fix
-While some experiment might be targeting them, operational metrics often act as guardrails and basic safety checks during experiments. +While some experiments might target them, operational metrics often act as guardrails and basic safety checks during experiments.
463-463: Remove grammatically incorrect colon.Line 463 places a colon after the preposition "with", which is grammatically incorrect. Either remove the colon and add a noun phrase, or restructure the sentence.
🔎 Proposed fix
-You want to optimise a landing page with: +You want to optimise a landing page with the following:Alternatively:
-You want to optimise a landing page with: +You want to optimise a landing page that includes:
233-233: Fix malformed bullet point text.Line 233 contains "-ehavior" which appears to be incomplete or corrupted text. This should likely be a complete bullet point about grounding hypotheses in user research and behaviour.
🔎 Proposed fix
--ehavior, prior data, or product theory. Avoid "spaghetti testing" — randomly trying ideas just to see what sticks. +- Ground hypotheses in user research, behaviour, prior data, or product theory. Avoid "spaghetti testing" — randomly trying ideas just to see what sticks.
32-32: Fix spelling error in A/B/n definition.Line 32 contains "refered" which should be "referred".
🔎 Proposed fix
-A/B/n experiments are also sometimes refered to as multi-variant. Not to be confused with [Multivariate Experiment](#multivariate-experiment). +A/B/n experiments are also sometimes referred to as multi-variant. Not to be confused with [Multivariate Experiment](#multivariate-experiment).
466-466: Add hyphens to compound modifier.Line 466 uses "call to action buttons" as a compound modifier before a noun, which requires hyphens.
🔎 Proposed fix
-- 2 different call to action buttons (C1, C2) +- 2 different call-to-action buttons (C1, C2)
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
docs/glossary.mdx(1 hunks)
🧰 Additional context used
🪛 LanguageTool
docs/glossary.mdx
[grammar] ~15-~15: If ‘type’ is a classification term, ‘A’ is not necessary. Use “type of”. (The phrases ‘kind of’ and ‘sort of’ are informal if they mean ‘to some extent’.)
Context: ...- ## A ### A/A experiment A special type of A/B experiment where users are randomly s...
(KIND_OF_A)
[style] ~18-~18: Would you like to use the Oxford spelling “randomization”? The spelling ‘randomisation’ is also correct.
Context: ...M), tracking bugs or unexpected bias in randomisation before you start testing real changes. ...
(OXFORD_SPELLING_Z_NOT_S)
[grammar] ~31-~31: If ‘type’ is a classification term, ‘A’ is not necessary. Use “type of”. (The phrases ‘kind of’ and ‘sort of’ are informal if they mean ‘to some extent’.)
Context: ...rate. ### A/B/n experiment A special type of A/B experiment that compares a control to...
(KIND_OF_A)
[style] ~41-~41: Would you like to use the Oxford spelling “generalizability”? The spelling ‘generalisability’ is also correct.
Context: ...eting can reduce sample size and affect generalisability. Example: Running an experiment on...
(OXFORD_SPELLING_Z_NOT_S)
[uncategorized] ~50-~50: Possible missing comma found.
Context: ...lue) is the current performance of your metric usually measured in AA experiment or a ...
(AI_HYDRA_LEO_MISSING_COMMA)
[duplication] ~97-~97: Possible typo: you repeated a word.
Context: ...fidence interval of [+0.5%, +4.1%], it means means that if you were to repeat the same exp...
(ENGLISH_WORD_REPEAT_RULE)
[uncategorized] ~99-~99: Possible missing comma found.
Context: ...the CI in about 95 of those. You could say “We are 95% confident that the [true ef...
(AI_HYDRA_LEO_MISSING_COMMA)
[grammar] ~137-~137: The verb ‘covariate’ does not usually follow articles like ‘a’. Check that ‘covariate’ is spelled correctly; using ‘covariate’ as a noun may be non-standard.
Context: ...using correlated pre-experiment data as a covariate. CUPED can significantly improve sen...
(A_INFINITIVE)
[uncategorized] ~139-~139: Use a comma before ‘so’ if it connects two independent clauses (unless they are closely connected and short).
Context: ...ED can significantly improve sensitivity so experiments reach conclusions faster or...
(COMMA_COMPOUND_SENTENCE_2)
[style] ~170-~170: You’ve already used the word ‘also’ once in your sentence, so using it again may be redundant.
Context: ... but also practically meaningful. It's also used in power calculations when designing ex...
(REDUNDANT_FILLER)
[style] ~184-~184: To elevate your writing, try using a synonym here.
Context: ...actions can distort results and make it hard to attribute observed effects to a sing...
(HARD_TO)
[uncategorized] ~196-~196: Possible missing comma found.
Context: ...ngful changes or underestimating effect sizes which leads to unreliable decisions. A...
(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~212-~212: Try moving the adverb to make the sentence clearer.
Context: ...irections; confirmatory experiments are designed to rigorously test a specific hypothesis. Mixing the two modes can lead to inf...
(SPLIT_INFINITIVE)
[grammar] ~231-~231: The verb ‘consider’ is used with the gerund form.
Context: ...coveries is essential. A few things to consider to reduce the risks of False Discoveries -ehavio...
(ADMIT_ENJOY_VB)
[uncategorized] ~248-~248: “an” (indefinite article before a vowel sound) seems less likely than “any”.
Context: ... significant, but only 60 of those have an actual true effects, then your False Di...
(AI_HYDRA_LEO_CP_AN_ANY)
[grammar] ~251-~251: After ‘it’, use the third-person verb form “exists”.
Context: ...Failing to detect a real effect when it exist; equivalent to a Type II error. False...
(IT_VBZ)
[uncategorized] ~289-~289: Possible missing comma found.
Context: ... horizon methods are conceptually simple but are not robust to unplanned peeking or ...
(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~293-~293: ‘making a decision’ might be wordy. Consider a shorter alternative.
Context: ...an experiment for exactly two weeks and making a decision only after both weeks have completed. ...
(EN_WORDINESS_PREMIUM_MAKING_A_DECISION)
[style] ~303-~303: ‘for lack of’ might be wordy. Consider a shorter alternative.
Context: ...is, allows the experiment to stop early for lack of effect — indicating that the treatment ...
(EN_WORDINESS_PREMIUM_FOR_LACK_OF)
[uncategorized] ~321-~321: Did you mean: “By default,”?
Context: ...ces stricter control over error rates. By default GST experiment in ABsmartly uses a bind...
(BY_DEFAULT_COMMA)
[uncategorized] ~322-~322: Use a comma before ‘but’ if it connects two independent clauses (unless they are closely connected and short).
Context: ...n ABsmartly uses a binding futility type but this can be changed during the setup. ...
(COMMA_COMPOUND_SENTENCE_2)
[style] ~334-~334: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...re a few well-timed looks are enough. Group Sequential Testing is the default metho...
(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
[duplication] ~340-~340: Possible typo: you repeated a word.
Context: ...tps://absmartly.com/gst) ### Guardrail metrics Metrics monitored to ensure experiments stay wi...
(ENGLISH_WORD_REPEAT_RULE)
[uncategorized] ~362-~362: Possible missing comma found.
Context: ...able prediction about the outcome of an experiment usually describing how a change (treatm...
(AI_HYDRA_LEO_MISSING_COMMA)
[grammar] ~435-~435: This phrase is duplicated. You should probably use “Metric variance” only once.
Context: ...ve differently at different scales ### Metric variance Metric variance refers to the amount of variability or ...
(PHRASE_REPETITION)
[style] ~458-~458: Would you like to use the Oxford spelling “optimize”? The spelling ‘optimise’ is also correct.
Context: ...best. They are useful when you want to optimise several components together, such as he...
(OXFORD_SPELLING_Z_NOT_S)
[formatting] ~459-~459: If the ‘because’ clause is essential to the meaning, do not use a comma before the clause.
Context: ...ntly more traffic than a simple A/B test, because traffic must be spread across many vari...
(COMMA_BEFORE_BECAUSE)
[typographical] ~459-~459: Use a comma after an introductory phrase.
Context: ...tions and the analysis is more complex. For this reason Multivariate experiments are not suppor...
(COMMA_INTRODUCTORY_WORDS_PHRASES)
[style] ~463-~463: Would you like to use the Oxford spelling “optimize”? The spelling ‘optimise’ is also correct.
Context: ... ABsmartly. Example: You want to optimise a landing page with: - 2 different he...
(OXFORD_SPELLING_Z_NOT_S)
[typographical] ~463-~463: Do not use a colon (:) before a series that is introduced by a preposition (‘with’). Remove the colon or add a noun or a noun phrase after the preposition.
Context: ...* You want to optimise a landing page with: - 2 different headlines (H1, H2) - ...
(RP_COLON)
[grammar] ~466-~466: It appears that two hyphens are missing.
Context: ...ero images (I1, I2, I3) - 2 different call to action buttons (C1, C2) A multivariate expe...
(CALL_TO_ACTION_HYPHEN)
[style] ~481-~481: If ‘chance’ means ‘possibility’, this phrase is redundant. Consider writing “chance”.
Context: ... that any observed difference is due to random chance. The null hypothesis is the foundation...
(RANDOM_CHANCE)
[uncategorized] ~502-~502: Possible missing preposition found.
Context: ...ence. --- ## O ### Observed effect The measured difference between treatment a...
(AI_HYDRA_LEO_MISSING_IN)
[grammar] ~504-~504: Did you mean “sample”?
Context: ...ble from the data, but it is subject to sampling variability. Example: Treatment ...
(IT_IS_JJ_TO_VBG)
[style] ~529-~529: Would you like to use the Oxford spelling “utilization”? The spelling ‘utilisation’ is also correct.
Context: ... Example: Error rate, latency, CPU utilisation or cache hit rate. ### Outliers Outlie...
(OXFORD_SPELLING_Z_NOT_S)
[duplication] ~531-~531: Possible typo: you repeated a word.
Context: ...CPU utilisation or cache hit rate. ### Outliers Outliers are data points that are significantly ...
(ENGLISH_WORD_REPEAT_RULE)
[duplication] ~549-~549: Possible typo: you repeated a word.
Context: ...xperimentation results. --- ## P ### Peeking Peeking refers to looking at experiment results...
(ENGLISH_WORD_REPEAT_RULE)
[uncategorized] ~550-~550: Possible missing comma found.
Context: ...ed, especially to check for statistical significance and making decisions based on those ea...
(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~559-~559: Use a comma before ‘and’ if it connects two independent clauses (unless they are closely connected and short).
Context: ...ome promissing results. This is peeking and you may be acting on a false discovery....
(COMMA_COMPOUND_SENTENCE)
[uncategorized] ~569-~569: Possible missing comma found.
Context: ... experiment is a requirement for making good reliable data-informed decisions. ABsm...
(AI_HYDRA_LEO_MISSING_COMMA)
[duplication] ~575-~575: Possible typo: you repeated a word.
Context: ... significance level. ### Pre-selection bias Bias introduced when the users who enter a s...
(ENGLISH_WORD_REPEAT_RULE)
[uncategorized] ~585-~585: The noun “decision-making” (= the process of deciding something) is spelled with a hyphen.
Context: ...e primary metric. This will be the main decision making metric. It is usually good practice to ...
(DECISION_MAKING)
[uncategorized] ~598-~598: Possible missing comma found.
Context: ...xperimentation is not a one-off activity but a core part of how the organisation bui...
(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~598-~598: Do not mix variants of the same word (‘organisation’ and ‘organization’) within a single text.
Context: ...off activity but a core part of how the organisation builds products. ### P-value The p...
(EN_WORD_COHERENCY)
[style] ~622-~622: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...p explain why a primary metric changed. Secondary metrics can also be used to reduce the ...
(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
[uncategorized] ~624-~624: Possible missing comma found.
Context: ...onitoring average order value and gross conversions while the primary metric is net convers...
(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~636-~636: Possible missing comma found.
Context: ...* Discounts shown only to the treatment group affecting reference prices for control ...
(AI_HYDRA_LEO_MISSING_COMMA)
[grammar] ~648-~648: The expression ‘how to spread’ requires the preposition ‘to’.
Context: ... ### Standard deviation A measure of how spread out or variable your data is. It tells ...
(HOW_TO_VERB)
[uncategorized] ~665-~665: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...effect of the planned size or larger. High power means you are less likely to miss real ...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
[uncategorized] ~666-~666: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ... likely to miss real improvements; very low power leads to many inconclusive or misleadin...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
[formatting] ~718-~718: Insert a comma before quoting reported speech: “states, “”…
Context: ...t. ### Twyman’s law A heuristic that states “the more surprising a result looks, the ...
(SAID_COMMA_SPEECH)
[style] ~727-~727: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...anges based on spurious results. ### Type II error Failing to reject the null h...
(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
[uncategorized] ~752-~752: Do not mix variants of the same word (‘standardise’ and ‘standardize’) within a single text.
Context: ...me traffic. --- ## Z ### Z-score A standardised value expressing how many standard devi...
(EN_WORD_COHERENCY)
There was a problem hiding this comment.
Actionable comments posted: 2
♻️ Duplicate comments (4)
docs/glossary.mdx (4)
728-728: Fix broken internal link anchor.Line 728 references
#flase-negativewhich contains a typo. The correct anchor should be#false-negative. This creates a broken internal link and was flagged previously but remains unfixed.🔎 Proposed fix
-Failing to reject the null hypothesis when it is false; also called a [false negative](#flase-negative). +Failing to reject the null hypothesis when it is false; also called a [false negative](#false-negative).
704-704: Fix broken internal link anchor.Line 704 references
#one-trailed-analysisbut the correct anchor should be#one-tailed-analysis(note the typo "trailed" vs "tailed"). This creates a broken internal link and was flagged previously but remains unfixed.🔎 Proposed fix
-Two-tailed tests are more conservative than [one-tailed](#one-trailed-analysis) tests: they require stronger evidence to detect an effect, +Two-tailed tests are more conservative than [one-tailed](#one-tailed-analysis) tests: they require stronger evidence to detect an effect,
558-558: Fix spelling error.Line 558 contains "promissing" which should be spelled "promising". This issue was flagged previously but remains unfixed.
🔎 Proposed fix
-On day 6, with only 40% of the data collected, you decide to stop early and ship because you see some promissing results. +On day 6, with only 40% of the data collected, you decide to stop early and ship because you see some promising results.
233-233: Fix double comma typo.Line 233 contains a double comma that should be a single comma.
🔎 Proposed fix
-- Ground hypotheses in user research, behaviour,, prior data, or product theory. Avoid "spaghetti testing" — randomly trying ideas just to see what sticks. +- Ground hypotheses in user research, behaviour, prior data, or product theory. Avoid "spaghetti testing" — randomly trying ideas just to see what sticks.
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
docs/glossary.mdx(1 hunks)
🧰 Additional context used
🪛 LanguageTool
docs/glossary.mdx
[grammar] ~15-~15: If ‘type’ is a classification term, ‘A’ is not necessary. Use “type of”. (The phrases ‘kind of’ and ‘sort of’ are informal if they mean ‘to some extent’.)
Context: ...- ## A ### A/A experiment A special type of A/B experiment where users are randomly s...
(KIND_OF_A)
[style] ~18-~18: Would you like to use the Oxford spelling “randomization”? The spelling ‘randomisation’ is also correct.
Context: ...M), tracking bugs or unexpected bias in randomisation before you start testing real changes. ...
(OXFORD_SPELLING_Z_NOT_S)
[grammar] ~31-~31: If ‘type’ is a classification term, ‘A’ is not necessary. Use “type of”. (The phrases ‘kind of’ and ‘sort of’ are informal if they mean ‘to some extent’.)
Context: ...rate. ### A/B/n experiment A special type of A/B experiment that compares a control to...
(KIND_OF_A)
[style] ~41-~41: Would you like to use the Oxford spelling “generalizability”? The spelling ‘generalisability’ is also correct.
Context: ...eting can reduce sample size and affect generalisability. Example: Running an experiment on...
(OXFORD_SPELLING_Z_NOT_S)
[uncategorized] ~50-~50: Possible missing comma found.
Context: ...lue) is the current performance of your metric usually measured in AA experiment or a ...
(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~97-~97: The preposition ‘of’ seems more likely in this position.
Context: ...the true effect would lie within the CI in about 95 of those. You could say “We a...
(AI_HYDRA_LEO_REPLACE_IN_OF)
[uncategorized] ~99-~99: Possible missing comma found.
Context: ...the CI in about 95 of those. You could say “We are 95% confident that the [true ef...
(AI_HYDRA_LEO_MISSING_COMMA)
[grammar] ~137-~137: The verb ‘covariate’ does not usually follow articles like ‘a’. Check that ‘covariate’ is spelled correctly; using ‘covariate’ as a noun may be non-standard.
Context: ...using correlated pre-experiment data as a covariate. CUPED can significantly improve sen...
(A_INFINITIVE)
[uncategorized] ~139-~139: Use a comma before ‘so’ if it connects two independent clauses (unless they are closely connected and short).
Context: ...ED can significantly improve sensitivity so experiments reach conclusions faster or...
(COMMA_COMPOUND_SENTENCE_2)
[style] ~170-~170: You’ve already used the word ‘also’ once in your sentence, so using it again may be redundant.
Context: ... but also practically meaningful. It's also used in power calculations when designing ex...
(REDUNDANT_FILLER)
[style] ~184-~184: To elevate your writing, try using a synonym here.
Context: ...actions can distort results and make it hard to attribute observed effects to a sing...
(HARD_TO)
[uncategorized] ~196-~196: Possible missing comma found.
Context: ...ngful changes or underestimating effect sizes which leads to unreliable decisions. A...
(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~212-~212: Try moving the adverb to make the sentence clearer.
Context: ...irections; confirmatory experiments are designed to rigorously test a specific hypothesis. Mixing the two modes can lead to inf...
(SPLIT_INFINITIVE)
[grammar] ~231-~231: The verb ‘consider’ is used with the gerund form.
Context: ...coveries is essential. A few things to consider to reduce the risks of False Discoveries - Groun...
(ADMIT_ENJOY_VB)
[typographical] ~233-~233: Two consecutive commas
Context: ...d hypotheses in user research, behaviour,, prior data, or product theory. Avoid “s...
(DOUBLE_PUNCTUATION)
[uncategorized] ~248-~248: “an” (indefinite article before a vowel sound) seems less likely than “any”.
Context: ... significant, but only 60 of those have an actual true effects, then your False Di...
(AI_HYDRA_LEO_CP_AN_ANY)
[uncategorized] ~289-~289: Possible missing comma found.
Context: ... horizon methods are conceptually simple but are not robust to unplanned peeking or ...
(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~293-~293: ‘making a decision’ might be wordy. Consider a shorter alternative.
Context: ...an experiment for exactly two weeks and making a decision only after both weeks have completed. ...
(EN_WORDINESS_PREMIUM_MAKING_A_DECISION)
[style] ~303-~303: ‘for lack of’ might be wordy. Consider a shorter alternative.
Context: ...is, allows the experiment to stop early for lack of effect — indicating that the treatment ...
(EN_WORDINESS_PREMIUM_FOR_LACK_OF)
[uncategorized] ~322-~322: Use a comma before ‘but’ if it connects two independent clauses (unless they are closely connected and short).
Context: ...n ABsmartly uses a binding futility type but this can be changed during the setup. ...
(COMMA_COMPOUND_SENTENCE_2)
[style] ~334-~334: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...re a few well-timed looks are enough. Group Sequential Testing is the default metho...
(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
[duplication] ~340-~340: Possible typo: you repeated a word.
Context: ...tps://absmartly.com/gst) ### Guardrail metrics Metrics monitored to ensure experiments stay wi...
(ENGLISH_WORD_REPEAT_RULE)
[uncategorized] ~362-~362: Possible missing comma found.
Context: ...able prediction about the outcome of an experiment usually describing how a change (treatm...
(AI_HYDRA_LEO_MISSING_COMMA)
[grammar] ~435-~435: This phrase is duplicated. You should probably use “Metric variance” only once.
Context: ...ve differently at different scales ### Metric variance Metric variance refers to the amount of variability or ...
(PHRASE_REPETITION)
[style] ~458-~458: Would you like to use the Oxford spelling “optimize”? The spelling ‘optimise’ is also correct.
Context: ...best. They are useful when you want to optimise several components together, such as he...
(OXFORD_SPELLING_Z_NOT_S)
[formatting] ~459-~459: If the ‘because’ clause is essential to the meaning, do not use a comma before the clause.
Context: ...ntly more traffic than a simple A/B test, because traffic must be spread across many vari...
(COMMA_BEFORE_BECAUSE)
[typographical] ~459-~459: Use a comma after an introductory phrase.
Context: ...tions and the analysis is more complex. For this reason Multivariate experiments are not suppor...
(COMMA_INTRODUCTORY_WORDS_PHRASES)
[style] ~463-~463: Would you like to use the Oxford spelling “optimize”? The spelling ‘optimise’ is also correct.
Context: ... ABsmartly. Example: You want to optimise a landing page with the following: - ...
(OXFORD_SPELLING_Z_NOT_S)
[style] ~481-~481: If ‘chance’ means ‘possibility’, this phrase is redundant. Consider writing “chance”.
Context: ... that any observed difference is due to random chance. The null hypothesis is the foundation...
(RANDOM_CHANCE)
[uncategorized] ~502-~502: Possible missing preposition found.
Context: ...ence. --- ## O ### Observed effect The measured difference between treatment a...
(AI_HYDRA_LEO_MISSING_IN)
[grammar] ~504-~504: Did you mean “sample”?
Context: ...ble from the data, but it is subject to sampling variability. Example: Treatment ...
(IT_IS_JJ_TO_VBG)
[style] ~529-~529: Would you like to use the Oxford spelling “utilization”? The spelling ‘utilisation’ is also correct.
Context: ... Example: Error rate, latency, CPU utilisation or cache hit rate. ### Outliers Outlie...
(OXFORD_SPELLING_Z_NOT_S)
[duplication] ~531-~531: Possible typo: you repeated a word.
Context: ...CPU utilisation or cache hit rate. ### Outliers Outliers are data points that are significantly ...
(ENGLISH_WORD_REPEAT_RULE)
[duplication] ~549-~549: Possible typo: you repeated a word.
Context: ...xperimentation results. --- ## P ### Peeking Peeking refers to looking at experiment results...
(ENGLISH_WORD_REPEAT_RULE)
[uncategorized] ~550-~550: Possible missing comma found.
Context: ...ed, especially to check for statistical significance and making decisions based on those ea...
(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~559-~559: Use a comma before ‘and’ if it connects two independent clauses (unless they are closely connected and short).
Context: ...ome promissing results. This is peeking and you may be acting on a false discovery....
(COMMA_COMPOUND_SENTENCE)
[uncategorized] ~569-~569: Possible missing comma found.
Context: ... experiment is a requirement for making good reliable data-informed decisions. The ...
(AI_HYDRA_LEO_MISSING_COMMA)
[duplication] ~575-~575: Possible typo: you repeated a word.
Context: ... significance level. ### Pre-selection bias Bias introduced when the users who enter a s...
(ENGLISH_WORD_REPEAT_RULE)
[uncategorized] ~585-~585: The noun “decision-making” (= the process of deciding something) is spelled with a hyphen.
Context: ...e primary metric. This will be the main decision making metric. It is usually good practice to ...
(DECISION_MAKING)
[uncategorized] ~598-~598: Possible missing comma found.
Context: ...xperimentation is not a one-off activity but a core part of how the organisation bui...
(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~598-~598: Do not mix variants of the same word (‘organisation’ and ‘organization’) within a single text.
Context: ...off activity but a core part of how the organisation builds products. ### P-value The p...
(EN_WORD_COHERENCY)
[style] ~622-~622: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...p explain why a primary metric changed. Secondary metrics can also be used to reduce the ...
(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
[uncategorized] ~624-~624: Possible missing comma found.
Context: ...onitoring average order value and gross conversions while the primary metric is net convers...
(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~636-~636: Possible missing comma found.
Context: ...* Discounts shown only to the treatment group affecting reference prices for control ...
(AI_HYDRA_LEO_MISSING_COMMA)
[grammar] ~648-~648: The expression ‘how to spread’ requires the preposition ‘to’.
Context: ... ### Standard deviation A measure of how spread out or variable your data is. It tells ...
(HOW_TO_VERB)
[uncategorized] ~665-~665: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...effect of the planned size or larger. High power means you are less likely to miss real ...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
[uncategorized] ~666-~666: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ... likely to miss real improvements; very low power leads to many inconclusive or misleadin...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
[formatting] ~718-~718: Insert a comma before quoting reported speech: “states, “”…
Context: ...t. ### Twyman’s law A heuristic that states “the more surprising a result looks, the ...
(SAID_COMMA_SPEECH)
[style] ~727-~727: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...anges based on spurious results. ### Type II error Failing to reject the null h...
(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
[uncategorized] ~752-~752: Do not mix variants of the same word (‘standardise’ and ‘standardize’) within a single text.
Context: ...me traffic. --- ## Z ### Z-score A standardised value expressing how many standard devi...
(EN_WORD_COHERENCY)
There was a problem hiding this comment.
Actionable comments posted: 3
♻️ Duplicate comments (4)
docs/glossary.mdx (4)
668-668: Fix spelling error."Typiocally" should be spelled "typically".
🔎 Proposed fix
-In AB testing, power is typiocally set to 80% meaning that 8 out of 10 times, the test will detect the planned effect. +In AB testing, power is typically set to 80% meaning that 8 out of 10 times, the test will detect the planned effect.
336-336: Fix incomplete comparative phrase – missing word.The sentence is missing the word "faster" in the comparative, making it grammatically incorrect and unclear.
🔎 Proposed fix
-It leads to making decisions up to 80% than with a more traditional [Fixed Horizon Experiment](#fixed-horizon-testing). +It leads to making decisions up to 80% faster than with a more traditional [Fixed Horizon Experiment](#fixed-horizon-testing).
558-558: Fix spelling error."Promissing" should be spelled "promising".
🔎 Proposed fix
-On day 6, with only 40% of the data collected, you decide to stop early and ship because you see some promissing results. +On day 6, with only 40% of the data collected, you decide to stop early and ship because you see some promising results.
704-704: Fix broken internal link anchor – typo in anchor text.The anchor text contains a typo: "trailed" should be "tailed". This will create a broken internal link.
🔎 Proposed fix
-Two-tailed tests are more conservative than [one-tailed](#one-trailed-analysis) tests: they require stronger evidence to detect an effect, +Two-tailed tests are more conservative than [one-tailed](#one-tailed-analysis) tests: they require stronger evidence to detect an effect,
🧹 Nitpick comments (1)
docs/glossary.mdx (1)
50-50: Add missing comma for clarity.Insert a comma after "metric" to improve readability: "the current performance of your metric, usually measured in..."
🔎 Proposed fix
-The baseline (or baseline value) is the current performance of your metric usually measured in AA experiment or a previous AB test. +The baseline (or baseline value) is the current performance of your metric, usually measured in AA experiment or a previous AB test.
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
docs/glossary.mdx(1 hunks)
🧰 Additional context used
🪛 LanguageTool
docs/glossary.mdx
[grammar] ~15-~15: If ‘type’ is a classification term, ‘A’ is not necessary. Use “type of”. (The phrases ‘kind of’ and ‘sort of’ are informal if they mean ‘to some extent’.)
Context: ...- ## A ### A/A experiment A special type of A/B experiment where users are randomly s...
(KIND_OF_A)
[style] ~18-~18: Would you like to use the Oxford spelling “randomization”? The spelling ‘randomisation’ is also correct.
Context: ...M), tracking bugs or unexpected bias in randomisation before you start testing real changes. ...
(OXFORD_SPELLING_Z_NOT_S)
[grammar] ~31-~31: If ‘type’ is a classification term, ‘A’ is not necessary. Use “type of”. (The phrases ‘kind of’ and ‘sort of’ are informal if they mean ‘to some extent’.)
Context: ...rate. ### A/B/n experiment A special type of A/B experiment that compares a control to...
(KIND_OF_A)
[style] ~41-~41: Would you like to use the Oxford spelling “generalizability”? The spelling ‘generalisability’ is also correct.
Context: ...eting can reduce sample size and affect generalisability. Example: Running an experiment on...
(OXFORD_SPELLING_Z_NOT_S)
[uncategorized] ~50-~50: Possible missing comma found.
Context: ...lue) is the current performance of your metric usually measured in AA experiment or a ...
(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~97-~97: The preposition ‘of’ seems more likely in this position.
Context: ...the true effect would lie within the CI in about 95 of those. You could say “We a...
(AI_HYDRA_LEO_REPLACE_IN_OF)
[uncategorized] ~99-~99: Possible missing comma found.
Context: ...the CI in about 95 of those. You could say “We are 95% confident that the [true ef...
(AI_HYDRA_LEO_MISSING_COMMA)
[grammar] ~137-~137: The verb ‘covariate’ does not usually follow articles like ‘a’. Check that ‘covariate’ is spelled correctly; using ‘covariate’ as a noun may be non-standard.
Context: ...using correlated pre-experiment data as a covariate. CUPED can significantly improve sen...
(A_INFINITIVE)
[uncategorized] ~139-~139: Use a comma before ‘so’ if it connects two independent clauses (unless they are closely connected and short).
Context: ...ED can significantly improve sensitivity so experiments reach conclusions faster or...
(COMMA_COMPOUND_SENTENCE_2)
[style] ~170-~170: You’ve already used the word ‘also’ once in your sentence, so using it again may be redundant.
Context: ... but also practically meaningful. It's also used in power calculations when designing ex...
(REDUNDANT_FILLER)
[style] ~184-~184: To elevate your writing, try using a synonym here.
Context: ...actions can distort results and make it hard to attribute observed effects to a sing...
(HARD_TO)
[uncategorized] ~196-~196: Possible missing comma found.
Context: ...ngful changes or underestimating effect sizes which leads to unreliable decisions. A...
(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~212-~212: Try moving the adverb to make the sentence clearer.
Context: ...irections; confirmatory experiments are designed to rigorously test a specific hypothesis. Mixing the two modes can lead to inf...
(SPLIT_INFINITIVE)
[grammar] ~231-~231: The verb ‘consider’ is used with the gerund form.
Context: ...coveries is essential. A few things to consider to reduce the risks of False Discoveries - Groun...
(ADMIT_ENJOY_VB)
[uncategorized] ~248-~248: “an” (indefinite article before a vowel sound) seems less likely than “any”.
Context: ... significant, but only 60 of those have an actual true effects, then your False Di...
(AI_HYDRA_LEO_CP_AN_ANY)
[uncategorized] ~289-~289: Possible missing comma found.
Context: ... horizon methods are conceptually simple but are not robust to unplanned peeking or ...
(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~293-~293: ‘making a decision’ might be wordy. Consider a shorter alternative.
Context: ...an experiment for exactly two weeks and making a decision only after both weeks have completed. ...
(EN_WORDINESS_PREMIUM_MAKING_A_DECISION)
[style] ~303-~303: ‘for lack of’ might be wordy. Consider a shorter alternative.
Context: ...is, allows the experiment to stop early for lack of effect — indicating that the treatment ...
(EN_WORDINESS_PREMIUM_FOR_LACK_OF)
[uncategorized] ~322-~322: Use a comma before ‘but’ if it connects two independent clauses (unless they are closely connected and short).
Context: ...n ABsmartly uses a binding futility type but this can be changed during the setup. ...
(COMMA_COMPOUND_SENTENCE_2)
[style] ~334-~334: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...re a few well-timed looks are enough. Group Sequential Testing is the default metho...
(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
[duplication] ~340-~340: Possible typo: you repeated a word.
Context: ...tps://absmartly.com/gst) ### Guardrail metrics Metrics monitored to ensure experiments stay wi...
(ENGLISH_WORD_REPEAT_RULE)
[uncategorized] ~362-~362: Possible missing comma found.
Context: ...able prediction about the outcome of an experiment usually describing how a change (treatm...
(AI_HYDRA_LEO_MISSING_COMMA)
[grammar] ~435-~435: This phrase is duplicated. You should probably use “Metric variance” only once.
Context: ...ve differently at different scales ### Metric variance Metric variance refers to the amount of variability or ...
(PHRASE_REPETITION)
[style] ~458-~458: Would you like to use the Oxford spelling “optimize”? The spelling ‘optimise’ is also correct.
Context: ...best. They are useful when you want to optimise several components together, such as he...
(OXFORD_SPELLING_Z_NOT_S)
[formatting] ~459-~459: If the ‘because’ clause is essential to the meaning, do not use a comma before the clause.
Context: ...ntly more traffic than a simple A/B test, because traffic must be spread across many vari...
(COMMA_BEFORE_BECAUSE)
[typographical] ~459-~459: Use a comma after an introductory phrase.
Context: ...tions and the analysis is more complex. For this reason Multivariate experiments are not suppor...
(COMMA_INTRODUCTORY_WORDS_PHRASES)
[style] ~463-~463: Would you like to use the Oxford spelling “optimize”? The spelling ‘optimise’ is also correct.
Context: ... ABsmartly. Example: You want to optimise a landing page with the following: - ...
(OXFORD_SPELLING_Z_NOT_S)
[style] ~481-~481: If ‘chance’ means ‘possibility’, this phrase is redundant. Consider writing “chance”.
Context: ... that any observed difference is due to random chance. The null hypothesis is the foundation...
(RANDOM_CHANCE)
[uncategorized] ~502-~502: Possible missing preposition found.
Context: ...ence. --- ## O ### Observed effect The measured difference between treatment a...
(AI_HYDRA_LEO_MISSING_IN)
[grammar] ~504-~504: Did you mean “sample”?
Context: ...ble from the data, but it is subject to sampling variability. Example: Treatment ...
(IT_IS_JJ_TO_VBG)
[style] ~529-~529: Would you like to use the Oxford spelling “utilization”? The spelling ‘utilisation’ is also correct.
Context: ... Example: Error rate, latency, CPU utilisation or cache hit rate. ### Outliers Outlie...
(OXFORD_SPELLING_Z_NOT_S)
[duplication] ~531-~531: Possible typo: you repeated a word.
Context: ...CPU utilisation or cache hit rate. ### Outliers Outliers are data points that are significantly ...
(ENGLISH_WORD_REPEAT_RULE)
[duplication] ~549-~549: Possible typo: you repeated a word.
Context: ...xperimentation results. --- ## P ### Peeking Peeking refers to looking at experiment results...
(ENGLISH_WORD_REPEAT_RULE)
[uncategorized] ~550-~550: Possible missing comma found.
Context: ...ed, especially to check for statistical significance and making decisions based on those ea...
(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~559-~559: Use a comma before ‘and’ if it connects two independent clauses (unless they are closely connected and short).
Context: ...some promising results. This is peeking and you may be acting on a false discovery....
(COMMA_COMPOUND_SENTENCE)
[uncategorized] ~569-~569: Possible missing comma found.
Context: ... experiment is a requirement for making good reliable data-informed decisions. The ...
(AI_HYDRA_LEO_MISSING_COMMA)
[duplication] ~575-~575: Possible typo: you repeated a word.
Context: ... significance level. ### Pre-selection bias Bias introduced when the users who enter a s...
(ENGLISH_WORD_REPEAT_RULE)
[uncategorized] ~585-~585: The noun “decision-making” (= the process of deciding something) is spelled with a hyphen.
Context: ...e primary metric. This will be the main decision making metric. It is usually good practice to ...
(DECISION_MAKING)
[uncategorized] ~598-~598: Possible missing comma found.
Context: ...xperimentation is not a one-off activity but a core part of how the organisation bui...
(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~598-~598: Do not mix variants of the same word (‘organisation’ and ‘organization’) within a single text.
Context: ...off activity but a core part of how the organisation builds products. ### P-value The p...
(EN_WORD_COHERENCY)
[style] ~622-~622: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...p explain why a primary metric changed. Secondary metrics can also be used to reduce the ...
(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
[uncategorized] ~624-~624: Possible missing comma found.
Context: ...onitoring average order value and gross conversions while the primary metric is net convers...
(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~636-~636: Possible missing comma found.
Context: ...* Discounts shown only to the treatment group affecting reference prices for control ...
(AI_HYDRA_LEO_MISSING_COMMA)
[grammar] ~648-~648: The expression ‘how to spread’ requires the preposition ‘to’.
Context: ... ### Standard deviation A measure of how spread out or variable your data is. It tells ...
(HOW_TO_VERB)
[uncategorized] ~665-~665: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...effect of the planned size or larger. High power means you are less likely to miss real ...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
[uncategorized] ~666-~666: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ... likely to miss real improvements; very low power leads to many inconclusive or misleadin...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
[formatting] ~718-~718: Insert a comma before quoting reported speech: “states, “”…
Context: ...t. ### Twyman’s law A heuristic that states “the more surprising a result looks, the ...
(SAID_COMMA_SPEECH)
[style] ~727-~727: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...anges based on spurious results. ### Type II error Failing to reject the null h...
(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
[uncategorized] ~752-~752: Do not mix variants of the same word (‘standardise’ and ‘standardize’) within a single text.
Context: ...me traffic. --- ## Z ### Z-score A standardised value expressing how many standard devi...
(EN_WORD_COHERENCY)
| Peeking refers to looking at experiment results before the test is completed, especially to check for statistical significance and | ||
| making decisions based on those early results without proper statistical adjustments. | ||
|
|
||
| Peeking inflates the [false positive rate](##false-discovery-rate-fdr), making it more likely that you'll incorrectly conclude a treatment is effective when it’s not. |
There was a problem hiding this comment.
Fix malformed internal link – remove extra hash.
The link anchor contains a double hash (##) which will break the internal navigation.
🔎 Proposed fix
-Peeking inflates the [false positive rate](##false-discovery-rate-fdr), making it more likely that you'll incorrectly conclude a treatment is effective when it's not.
+Peeking inflates the [false positive rate](#false-discovery-rate-fdr), making it more likely that you'll incorrectly conclude a treatment is effective when it's not. 🤖 Prompt for AI Agents
In docs/glossary.mdx around line 553, the internal link uses a double hash
("##false-discovery-rate-fdr") which breaks navigation; edit the link to use a
single hash ("#false-discovery-rate-fdr") so it becomes a valid internal anchor,
and verify the target heading ID exists or adjust the anchor to match the actual
heading.
|
|
||
| ### Sample size | ||
| The number of visitors included in an experiment. | ||
| Sample size, together with [variance](#variance) and [effect size](#effect-size), determines [power](##experimentation-power) and the time needed to reach a conclusion. |
There was a problem hiding this comment.
Fix malformed internal link – remove extra hash.
The link anchor contains a double hash (##) which will break the internal navigation.
🔎 Proposed fix
-Sample size, together with [variance](#variance) and [effect size](#effect-size), determines [power](##experimentation-power) and the time needed to reach a conclusion.
+Sample size, together with [variance](#variance) and [effect size](#effect-size), determines [power](#experimentation-power) and the time needed to reach a conclusion. 📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| Sample size, together with [variance](#variance) and [effect size](#effect-size), determines [power](##experimentation-power) and the time needed to reach a conclusion. | |
| Sample size, together with [variance](#variance) and [effect size](#effect-size), determines [power](#experimentation-power) and the time needed to reach a conclusion. |
🤖 Prompt for AI Agents
In docs/glossary.mdx around line 618, the internal link to the "power" section
uses a malformed anchor with a double hash ("##experimentation-power"); replace
it with a single hash anchor ("#experimentation-power") so the internal
navigation link points correctly to the "power" heading.
| Type I errors lead to rolling out ineffective changes based on spurious results. | ||
|
|
||
| ### Type II error | ||
| Failing to reject the null hypothesis when it is false; also called a [false negative](#false-negative). |
There was a problem hiding this comment.
Fix broken internal link anchor – typo in anchor text.
The anchor text contains a typo: "flase" should be "false". This will create a broken internal link.
🔎 Proposed fix
-Failing to reject the null hypothesis when it is false; also called a [false negative](#flase-negative).
+Failing to reject the null hypothesis when it is false; also called a [false negative](#false-negative). Committable suggestion skipped: line range outside the PR's diff.
🤖 Prompt for AI Agents
In docs/glossary.mdx around line 728 the internal anchor link text contains a
typo ("flase-negative") causing a broken link; correct the misspelling to
"false-negative" so the anchor/href matches the actual target ID (update the
link from #flase-negative to #false-negative and ensure the corresponding
heading/id uses the same corrected spelling).
Added new documentation for events
Summary by CodeRabbit
Documentation
Chores
✏️ Tip: You can customize this high-level summary in your review settings.