Personalized System Instructions in boisestate.ai

• Define system instructions as persistent prompts that define AI's role, expertise, tone, and boundaries for an entire conversation.

• Observe how system instructions transform generic AI responses into precisely calibrated support that meets users exactly where they are.

• Use a concrete example (e.g., explaining the concept of p-values in research) to compare how system instructions influence the responses to the same question:

  • Without system instructions (generic, middle-of-the-road)

  • For beginners (encouraging, intuitive, jargon-free)

  • For intermediate users (consultative, practical, nuanced)

  • For experts (collegial, sophisticated, research-focused)

• Explain how language, depth, examples, and tone automatically adapt to serve freshmen, graduate students, and faculty researchers with equally effective but completely different responses

Applying & Creating: Learners use boisestate.ai to create system instructions.

Analyzing & Evaluating: Learners analyze and assess how system instructions influence AI responses.

Where boisestate.ai use is appropriate and a core component of work, instructors, staff, or students might engage with system instructions to generate a set of guidelines for AI responses calibrated specific to their need:

• Explaining complex concepts

• Personas

• Interviews

• Planning

By adapting the lesson example to fit the context and people who would be using the system instructions, boisestate.ai can support reflection and AI literacy.

When you start a conversation with an AI model on boisestate.ai, there's an invisible guide shaping how the AI responds to you. These guides are called system instructions—persistent prompts that define AI’s role, expertise, tone, and boundaries for an entire conversation.

Think of system instructions as the difference between asking a general question to a crowd versus asking a specific expert. Without system instructions, you're shouting into a room and getting a generic answer. With system instructions, you're consulting with a specialist who understands exactly what you need.

Why System Instructions Matter in Higher Education

In an academic setting, context is everything. A freshman asking about research methods needs a completely different response than a doctoral candidate working on their dissertation. A student seeking help with a paper needs guidance that promotes learning, not answers that enable academic dishonesty. System instructions make these crucial distinctions possible.

System instructions allow you to:

• Customize boisestate.ai ’s expertise level to match your audience

• Ensure responses align with institutional policies and pedagogy

• Maintain consistent tone and approach across all interactions

• Set appropriate boundaries (when to defer to humans, what to avoid)

• Reference Boise State-specific resources, terminology, and procedures

Response

A p-value is a statistical measure that helps you determine the significance of your results. Here's what you need to know:

Definition: The p-value represents the probability of obtaining results at least as extreme as those observed, assuming the null hypothesis is true.

Interpretation:

• A p-value less than 0.05 (5%) is typically considered statistically significant

• This threshold means there's less than a 5% chance your results occurred by random chance

• Lower p-values indicate stronger evidence against the null hypothesis

Common significance levels:

• p < 0.05: Significant

• p < 0.01: Highly significant

• p < 0.001: Very highly significant

Important considerations:

1. P-values don't tell you the size or importance of an effect

2. Statistical significance ≠ practical significance

3. Always consider the context of your research

4. Effect sizes and confidence intervals provide additional important information

Example: If you're testing whether a new teaching method improves test scores and you get p = 0.03, this suggests the improvement is unlikely to be due to chance alone.

Would you like me to explain how this applies to your specific research context?

System Instructions

Create system instructions that tell boisestate.ai it's helping undergraduate students who are brand new to statistics and may have math anxiety.

Response

Great question! The p-value is one of those concepts that seems mysterious at first, but it's actually pretty intuitive once you break it down.

Think of it like this:

Imagine you flip a coin 10 times and get 9 heads. You'd probably think, "Wait, is this coin rigged?" because that seems really unlikely to happen by pure chance with a fair coin. The p-value is basically answering the question: "If nothing special is going on, how weird is my result?"

In research terms:

• You have a hypothesis (like "this new study technique improves test scores")

• You collect data

• The p-value tells you: "If this technique actually does NOTHING, what are the odds I'd see results this strong just by random luck?"

The common rule of thumb:

• p < 0.05 (less than 5%) = "This is unlikely to be just random chance—something's probably going on here!"

• p > 0.05 = "This could easily happen by chance—we don't have strong evidence for an effect"

Important: A small p-value doesn't tell you HOW MUCH of an effect there is, just whether the effect is probably real vs. random noise.

Does this make sense? What kind of research are you working on? I can give you a more specific example related to your project!

System Instructions

Adapt the system instructions for graduate students with basic statistics training who are working on thesis research. They need more sophistication but also practical guidance for their projects.

Response

The p-value is the probability of observing data as extreme as (or more extreme than) what you actually observed, assuming the null hypothesis is true. Let me unpack what that means and address some common interpretation issues.

What it tells you: The p-value quantifies the compatibility between your data and the null hypothesis. A p-value of 0.03 means "if there's truly no effect, I'd see results this extreme only 3% of the time by random sampling variability."

Critical distinctions to keep in mind:

• It's NOT the probability that the null hypothesis is true (that's a Bayesian posterior)

• It's NOT the probability you've made a wrong decision

• It's NOT the probability your results occurred by chance

• It DOES depend on sample size—large samples can yield tiny p-values for trivial effects

Practical interpretation guidance:

• Always report effect sizes alongside p-values—a p < 0.001 with Cohen's d = 0.15 tells a very different story than one with d = 0.85

• Consider your statistical power—was your study adequately powered to detect meaningful effects?

• The p = 0.05 threshold is conventional, not magical—context matters

• Distinguish between statistical significance and practical/clinical significance

For your research: What's your study design and sample size? Those factors influence how you should interpret and report your p-values. Also, are you planning multiple comparisons? That affects your interpretation strategy.

System Instructions

Modify the system instructions for faculty researchers and advanced scholars who engage with methodological literature and need sophisticated consultation.

Response

The p-value is P(data | H₀)—the probability of observing a test statistic at least as extreme as yours under the null model. As you know, this frequentist framework has been increasingly scrutinized in the replication crisis and methodological reform discussions.

Key considerations for your work:

Interpretational constraints:

• P-values don't provide P(H₀ | data), despite common misinterpretation—that requires priors (Bayesian framework)

• The dichotomous p < .05 decision rule has minimal theoretical justification (see Wasserstein & Lazar, 2016 ASA statement)

• Consider reporting exact p-values and confidence intervals rather than just "p < .05"

Methodological alternatives to consider:

• Effect sizes with CIs: Emphasize estimation over testing—particularly relevant given NHST's sensitivity to sample size

• Bayesian approaches: Bayes factors quantify relative evidence for competing hypotheses without arbitrary thresholds

• Equivalence testing: TOST procedures when your research question involves demonstrating similarity rather than difference

• Specification curve analysis: If you're concerned about researcher degrees of freedom and analytic flexibility

Current best practices:

• Pre-register primary analyses to distinguish confirmatory from exploratory

• Correct for multiple comparisons if conducting family-wise tests (Bonferroni, FDR, etc.)

• Report statistical power/sensitivity analyses, especially for null results

• Consider your inferential goals—are you testing a point null (unlikely to be precisely true) or a substantive hypothesis?

What's your specific context? Are you dealing with multiple testing issues, concerned about Type M/S errors in underpowered analyses, or navigating reviewer concerns about marginal p-values? The optimal approach depends on your research question and field conventions.