Ask the supplier to demonstrate one allowed request, one denied request, a revoked permission, and a failed connector. Record the observed result and the exact configuration tested. Architecture diagrams should explain the test, not replace it.
Define the Actual Security Boundary
Record where source data, embeddings, prompts, responses, credentials, and logs are stored or processed. Identify all providers and distinguish configurable controls from controls demonstrated in the offered setup.
Request Demonstrations of Access Behavior
Test an allowed request, a denied document, revoked access, and an exported answer or citation. Use two test users with different access; redact the evidence before sharing.
A convincing access test includes a reason why the request should be denied. Suppose a test employee can read general operating guidance but cannot read personnel records. The supplier should show which identity the application uses and where the restriction is enforced. A response saying that information is unavailable is useful evidence only if the underlying retrieval and tool activity also respect that restriction. Otherwise, the interface may hide a result after restricted information has already entered a prompt, response record, or diagnostic log.
Test Retrieved Instructions and Tool Permissions
Put a harmless adversarial instruction in an approved test document and observe whether it changes behavior. Confirm that permissions and consequential actions are checked outside the model.
Example: A test document instructs the assistant to send a summary to an unrelated address. The system should not gain that ability from the document.
Review Logging and Operational Ownership
Ask what is logged, who can view it, how long it is retained, and who handles a suspected incident. List unresolved issues with owners, tested configuration, remediation evidence, and a retest date.
Discuss the implementation scope with Sprinklenet. A useful starting point: a review of the proposed AI boundary and a documented test plan.
References
The recommendations above are Sprinklenet’s practical guidance. Technical context: OWASP Prompt Injection, OWASP Excessive Agency, OpenTelemetry Sensitive Data Handling.
