Mastering AI Legal Prompts: A Guide for In-House Counsel

There’s no mistaking the fact that AI is transforming in-house legal departments. The secret to unlocking AI’s potential for your legal workflow is not just choosing the right tool, but also pairing that tool with a powerful prompt. The right prompt can make all the difference when it comes to getting the most out of AI tools, whether you're drafting documents, streamlining workflows, or turbocharging your team’s decision-making.

In a recent webinar hosted by the L Suite, “Prompting with Precision: Leveraging AI Responsibly as In-House Counsel”, member panelists Alex Rindels (General Counsel, Jasper), and Jason Sitomer (General Counsel, TIFIN) shared strategies to get the most effective result from their AI tool. The panel was moderated by Tommie Tavares-Ferreira (Chief Strategy Officer, Lawtrades), with thanks to our sponsor Lawtrades for making this event possible. Note these insights are paraphrased for brevity and clarity.

Let's dive into the five best practices they rely on every time they prompt:

Give the AI a Role To Play

If you don’t specify the AI’s role, the model will choose on its own. For many of the general-purpose models like ChatGPT and Claude, the default profile is an eager-to-please personal assistant, which is skewed toward delighting you rather than applying relevant subject matter expertise.

To make each prompt go further, the panelists recommend you tell AI to act not just as an attorney, but as an attorney with the specific subject matter expertise you need.

Pro tip: After asking the model to play the role of a lawyer from the relevant practice area, treat the AI’s work product like you would that of a junior lawyer. Trust but verify.

Example: Summarize NY DFS 500 Regulations Effectively

You are the general counsel of a high-growth US company. Act as a highly knowledgeable and authoritative expert on the New York Department of Financial Services (NY DFS) Cybersecurity Regulation (23 NYCRR Part 500).

Your task is to provide comprehensive and accurate information regarding NY DFS 500. Specifically, you must:

• Summarize key provisions and obligations.

• Identify relevant deadlines and compliance dates.

• Explain the scope and applicability of the regulation.

• Clarify common ambiguities or misinterpretations.

• Discuss the implications of non-compliance.

• Provide guidance on best practices for achieving compliance, based solely on the regulation itself and other primary sources or reputable law firm publications from the last year.

• Cross-reference related sections within the regulation where appropriate.

Always:

• Cite the specific section of NY DFS 500 when providing information.

• Maintain a professional and objective tone.

• Emphasize that users should consult with their own counsel for definitive legal advice.

Ask AI To Write Out Its Reasoning in IRAC Format

We all remember IRAC with varying levels of fondness from law school. The structure is: Issue, Rule, Application, Conclusion. If you ask AI to specify each of these aspects of its analysis, you can easily diagnose whether AI is actually answering the question you care about, and whether it has made any mistakes during its chain of reasoning.

Pro tip: If you don’t have access to a legal-specific AI, models that are designed for coding perform very well in deductive legal reasoning, though the writing style may have less finesse.

Give a Lot of Context, but Don’t Lead the Witness

Have you ever been in a situation where sales asks legal if they can use a vendor’s contract, but they don’t explain the arrangement or even share a copy of that document? Don’t ask your AI questions without context. The more background you can give the tool before asking a question, the better answers you will receive.

The answers will improve further if you ask open-ended questions instead of leading ones, as the latter can sway the outcome. If you wind up in a scenario where you’ve given a lot of context and asked open-ended questions, but are still not satisfied with the answer, starting a fresh instance with a new prompt can help you troubleshoot to get a better response.

Pro tip: If you’ll need to use the same context multiple times, you may be able to add this information to your model so it applies to future questions as well.

Use AI to Improve Your Prompt

Some legal AI providers, such as White Shoe, include a feature that helps you improve your prompt. If your model doesn’t have that feature, you can ask it specifically for help with prompting. Alternatively, consider taking the output from one model and feeding it into another to cross-check the reasoning. It can be helpful to ask the second AI to argue against the first one’s answer or to check it for errors.

Pro tip: Create a shared prompt library with your team’s best prompts to boost collective learning and efficiency.

Example: Refine AI Answer by Adjusting Key Concepts

Based on your previous response, please identify the key areas or concepts in that reply that I can adjust or refine to change the outcome. Highlight any specific words, phrases, or ideas that, if clarified or altered, would significantly impact the direction or depth of your next response. Please also suggest potential “dials” or “knobs” I can turn, such as:

• Increasing or decreasing the level of detail or specificity

• Shifting the tone (e.g., more formal, casual, technical, creative, etc.)

• Focusing on a different aspect of the topic (e.g., a narrower or broader scope)

• Elaborating on particular areas that were mentioned but not fully explored

• Providing examples, analogies, or further explanations to clarify the response

Ask Questions That Can't Be Easily Falsified

Treating AI like a subject matter expert can be problematic if you don’t have enough knowledge to identify whether the answers are accurate. If you are asking about a topic you don’t know very well, your first goal is to figure out what you don’t know in order to ask a better question.

The most useful responses in this case are ones you can verify easily, by comparing the output against the original source and reviewing the AI’s reasoning.

Pro tip: Even though AI-generated citations can be wrong, ask for them anyway to help you validate the model’s output.

Example: Summarize Vendor AI Training Rights Clearly

# AI Model Training Rights Summary (Vendor DPA)

**Document:** [ATTACHED VENDOR_DPA_DOCUMENT]

---

## Task

Review the attached DPA and extract any provisions related to **AI model training**, including:

- Use of customer data for training/fine-tuning

- Aggregation, anonymization, or derived data rights

- Data retention or secondary use for ML/AI

---

## Output Instructions

1. **Answer clearly:**

A. Does the vendor reserve rights to use customer data for AI model training?

B. Are there any limits (e.g., anonymization, opt-out, restrictions)?

C. If not addressed, state: *“No model training rights specified in this DPA.”*

2. **Cite precisely:** Include section number + heading (e.g., *Section 4.2 – Use of Data*).

3. **Style:**

- Keep under **200 words**

- Use **plain, non-legal language**

- Write in a **single concise paragraph**

Putting It Into Practice: The AI Super-Prompt Framework

During the webinar, a sample “super-prompt” was shared that can be customized to fit a number of common legal prompting situations. Here is a summary of the template includes that can get you started with more effective prompting:

• Description of the topic or document and the type of request (e.g., document review, legal analysis, research question)

• Research and reasoning requirements:

  • Include relevant laws, regulations, and precedents, and citations of the relevant paragraph/section numbers or websites

  • For document review: analyze each section separately, and flag problematic language with specific recommendations for improvement

  • For research: prioritize authoritative primary sources and note any areas where information is limited or conflicting

  • Provide a critical analysis of strengths, weaknesses, risks, and opportunities

  • Share a list of the assumptions and limitations in its analysis, as well as counterarguments and business implications

• Response format instructions:

  • Begin with a 3-5 bullet point executive summary

  • Include clear headers and subheaders

  • Bold key findings and recommendations

  • For document review: list issues with bolded risk categories. (Specify the exact risk categories your AI should select from)

  • Use bullet points for lists of considerations

  • Conclude with a “next steps” section that has actionable recommendations

Guardrails and Precautions for Responsible Use of AI

Even with the best prompt, AI does introduce its own set of risks that may not be present elsewhere in your tech stack. Many of the best practices above will help you navigate risks such as inaccurate or incomplete outputs.

When it comes to more legal-specific risks such as jeopardizing confidential or sensitive information, IP protectability, and attorney-client privilege, there are some additional mitigation steps you can take. Here are a few strategies from our webinar panelists on how to manage the risk associated with AI tools:

• Align with InfoSec first and foremost before adding a new AI tool to the stack

• Keep models from training on your data by using enterprise accounts wherever possible (and specifying this in your agreements), or changing settings in the case where you have to use personal accounts

• Prioritize tools that will encrypt your data. Certain legal AI providers do this automatically so the LLM does not even see company information if you input it. Attorney-client privilege is much more likely to be maintained in such cases

• Redact sensitive information from your prompt. For example, rename a company “Acme, Inc.” or “[Company 1]” before including it in a prompt, especially for public tools like ChatGPT or Claude

• Ensure attorney review of outputs for legal materials for both legal and ethical reasons

• Consider creating an AI use policy. Whether your company needs a formal policy depends on how much sensitive data it handles. For smaller companies or those with a lower risk profile, informal consensus to adhere to risk guidelines such as these may be enough

While there is risk involved in adding AI tools to legal workflows, our webinar panelists agreed this incremental risk is the same as bringing on any other vendor, and should be managed accordingly.

Final Thoughts

AI is already reshaping in-house legal departments, but the tool is only as powerful as the prompts that go into it. By asking AI to: play a specific role, write out its reasoning, answer open-ended falsifiable questions, and improve your prompts, you can get high-quality responses comparable to a star junior lawyer in a fraction of the time.

Great prompting isn’t just about keeping up with the next wave of technological change, but it’s an essential part of being a great lawyer and leader in this new digital era.

The L Suite community has resources to help your in-house team leverage generative AI effectively, including the Generative AI Playbook, on-demand content, and a library of real-world prompts available only to members.