What personality-conditioned AI agents actually demonstrate for cyber ranges: generated schedules, prompt-order bias and the next tests for realistic activity.
Select a figure to open it at full size.
A defensive cyber range needs believable everyday activity. Browsing, meetings, documents and breaks provide the background against which security teams investigate suspicious behavior. Replaying the same tidy script can produce an environment that feels artificial.
SANDMAN explores whether language models can generate more varied activity by giving simulated users different personality profiles. The research tests a useful first step: whether those profiles change generated daily agendas in interpretable ways.
It does not yet establish that the resulting agents fool attackers, improve incident detection or increase the realism of an entire organization. Understanding that boundary makes the paper more actionable for teams deciding what to build and measure next.
Newsham, Hyland and Prince describe an architecture with memory, planning and action components. A profile and stored context inform a daily plan. A decision engine and task generator then translate the plan into tasks, while channels connect those tasks to software activity.
The memory component distinguishes background knowledge, previous events and working context. The planning component combines an LLM with an engine; the action component carries tasks into application-specific channels. This separation makes it possible to ask where realism breaks down: in the plan, the timing, the application behavior or the connection between them.
The personality settings use the five-factor model: openness, conscientiousness, extraversion, agreeableness and neuroticism. The researchers prompt stronger or weaker expressions of a trait using descriptions, then observe the model’s responses and schedules. These are controls over generated behavior, not evidence that a model has a human personality.
The agenda experiments use GPT-3.5-turbo at a temperature of 0.7 and generate 500 schedules for each condition. Personality-questionnaire evaluations are repeated five times. Those experimental units should not be mistaken for 500 human participants or five deployed organizations.
The original duration table shows one concrete effect. Work tasks in the neutral condition average 63.9 minutes. Under the higher-conscientiousness condition, the average rises to 85.1 minutes. Coffee tasks move in the opposite direction, from 40.9 to 32.1 minutes. Under lower conscientiousness, coffee tasks average 49.6 minutes.
The mechanism is understandable: changing the profile changes how the model allocates time among activities. The result is not merely a different writing style for the same schedule.
But this is only one dimension of plausibility. A simulated user could receive an appropriately work-heavy agenda and still browse too mechanically, create implausible documents or repeat identical actions. An agenda generator therefore needs a second layer of evaluation at the activity level.
The full table also shows why selecting one appealing row is inadequate. Traits affect many tasks, with considerable variation. The researchers observe interactions between trait settings rather than perfectly independent controls. A “conscientiousness” adjustment cannot safely be assumed to leave every other aspect of behavior unchanged.
The researchers investigate whether the order of tasks supplied to the model influences the order it generates. It does. System instructions and randomizing the input list reduce this bias, but do not remove it completely.
That is important for deception research. If many supposed users share artifacts introduced by the prompt, adding different personality labels may still leave a recognizable synthetic pattern. Repeated schedules should therefore be checked for common ordering and timing signatures, not just plausible individual examples.
The paper also presents the most frequent task at each time slot. That display summarizes many generated days; it is not necessarily one complete schedule produced by a single agent. Reading it as an observed person’s routine would overstate what the analysis shows.
The evaluated work centers on initial agenda generation and individual agents. It does not demonstrate sustained social coordination between agents or an adversarial test in which observers try to distinguish these users from people. Those are consequential next tests, not details a deployment team can assume away.
For a range operator, the useful next step is to compare three sources of benign activity: the existing scripted baseline, randomized scripts and profile-conditioned schedules. Use the same applications and task inventory so the comparison isolates the scheduling approach as far as possible.
Then evaluate both the plan and its execution. Does the activity stay within working hours? Are task transitions possible? Do repeated users share suspiciously identical sequences? Can reviewers recognize the generated condition from its artifacts? Keep the exercise inside an authorized, isolated environment.
A personality-conditioned generator earns its complexity if it creates useful variation without sacrificing coherence or control. If randomized scripts achieve the same realism for less operational effort, that is a valuable result too.
CMU SEI’s GHOSTS provides a practical baseline for generating non-player-character activity in cyber ranges, including clients, activity timelines and supporting APIs. It can help separate the problem of executing activity from the experiment of generating agendas.
GHOSTS is not a drop-in implementation of this SANDMAN paper. Use a small set of existing timelines first, then translate generated agendas into the same supported actions. Reject impossible or unsupported tasks before execution, retain the original plan and log what actually ran.
Measure how much correction the generated timelines require and whether they produce a wider range of credible traces. The architecture discussed in our autonomous-agent article explains why plans, tool execution and persistent state need separate checks.
SANDMAN offers evidence that personality prompting can influence synthetic agendas. That is a useful controllability result, and the paper’s attention to ordering bias makes it more informative than a collection of convincing examples.
The deployment question remains whether those agendas produce believable, controllable activity over time. Build that comparison before claiming better deception. Personality is an input to the simulation; realism is an outcome that still has to be measured.
Inducing Personality in LLM-Based Honeypot Agents: Measuring the Effect on Human-Like Agenda Generation, Newsham, Hyland and Prince. Version 1, March 25, 2025. Original Figure 1 and Table 4 are included above.
Examine how relevance and trust affect responses to AI-generated phishing messages.
Extend safety testing to the memory, tools and actions surrounding an AI model.
