From 9b5f942eacbeb7115c7bce85205964943b8e4f90 Mon Sep 17 00:00:00 2001 From: Frank Xu Date: Wed, 24 Jul 2024 17:30:16 -0400 Subject: [PATCH] revise political analysis --- .../email_analysis_political_insight.ipynb | 266 +++++++++++++++--- 1 file changed, 222 insertions(+), 44 deletions(-) diff --git a/AI4Forensics/CKIM2024/HillaryEmails/email_analysis_political_insight.ipynb b/AI4Forensics/CKIM2024/HillaryEmails/email_analysis_political_insight.ipynb index 418e720..6970614 100644 --- a/AI4Forensics/CKIM2024/HillaryEmails/email_analysis_political_insight.ipynb +++ b/AI4Forensics/CKIM2024/HillaryEmails/email_analysis_political_insight.ipynb @@ -39,6 +39,184 @@ "- Can we use DSPy?" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Step 0: Download and check summarized email dataset " + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "! wget -q https://raw.githubusercontent.com/frankwxu/digital-forensics-lab/main/AI4Forensics/CKIM2024/HillaryEmails/results_email_summary.txt" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "No information related to Israel or Israeli affairs was found in the provided email.\n", + "12:30 Israeli PM Netanyahu\n", + "- Dennis Ross recently visited Israel and will share information with the recipient before upcoming meetings.\n", + "There is no mention of Israel or Israeli in the email provided.\n", + "**Subject:** The Vice President's Residence\n", + "\n", + "**Date:** N/A\n", + "\n", + "**Summary:**\n", + "\n", + "- 9:00 am: Bilateral meeting with Israeli President Shimon Peres at the Omni Shoreham Hotel.\n", + "This email does not mention Israel or Israeli-related topics.\n", + "No information related to Israel or Israeli is present in the provided email.\n", + "In a discussion about arrangements on settlements with Israel, two options are presented:\n", + "\n", + "1. Describe it as an agreement, which would raise concerns about legitimizing Israeli activity in the West Bank and triggering complaints from Arabs and Palestinians.\n", + "\n", + "2. The administration could acknowledge progress made, express differences with the Israeli government on their intention to complete housing units, and stress that substantial progress warrants the resumption of peace talks and Arab steps toward Israel for a two-state solution. This option maintains credibility and allows for forward movement.\n", + "- Israel strongly expressed to the US their concern about a potential UNGA vote on the peace process.\n", + "- George Mitchell, the US Special Envoy for Middle East Peace, reported this concern to Hillary Clinton and President Obama.\n", + "- Mitchell planned to inform Rahm Emanuel, White House Chief of Staff, and Dan Shapiro, US Ambassador to Israel, about Israel's position.\n", + "Sure, here is a concise summary of the email you provided, focusing on Israel-related information:\n", + "\n", + "**Email Summary**\n", + "\n", + "**From:** Hillary Clinton\n", + "**To:** N/A\n", + "**Date:** N/A\n", + "**Subject:** N/A\n", + "**Israel-Related Information:**\n", + "\n", + "- Phone call with Israeli Foreign Minister Avigdor Lieberman scheduled for 2:15pm.\n", + "I cannot find any information about Israel in the provided email.\n", + "There is no mention of Israel or Israeli in the provided text.\n", + "**Summary:**\n", + "\n", + "* Statement regarding Israel-Palestinian conflict to be released in sender's name.\n", + "* Statement supports good-faith negotiations for a two-state solution.\n", + "* Palestinian goal: Independent state based on 1967 lines with agreed swaps.\n", + "* Israeli goal: Jewish state with secure borders reflecting subsequent developments and meeting security requirements.\n", + "* Commitment to achieving a two-state solution with peace and security is unwavering.\n", + "- 2:00pm SECURE CALL w/ISRAELI PM BINYAMIN NETANYAHU\n", + "- 2:30pm PHONE CALL w/ISRAELI DEFENSE MINISTER EHUD BARAK\n", + "The provided email doesn't contain any information related to Israel or Israeli.\n", + "No information related to Israel found in the provided email content.\n", + "**Summary:**\n", + "\n", + "On October 28, 2009, Palau and Israel joined the United States in voting against a UN resolution calling for an end to the US embargo on Cuba.\n", + "On Friday, Steinberg reportedly reprimanded Israeli Ambassador to the US Michael Oren over the country's conduct during Vice President Biden's visit.\n", + "This email does not pertain to Israel.\n", + "No relevant information about Israel was included in the provided email.\n", + "FYI. Israelis spinning up a storm, starting with plane briefing on the way over.\n", + "No information related to Israel or Israelis was found in the provided email.\n", + "This email discusses Israel's settlement freeze and construction in the West Bank. Key points include:\n", + "\n", + "- Ongoing construction in Israeli settlement of Har Homa exposes Netanyahu's settlement freeze as an illusion.\n", + "- Right-wing parties in Israel's coalition government are pressuring Netanyahu to block the renewal of the settlement freeze policy, which is set to expire on September 26.\n", + "- Israeli Minister of Public Affairs and the Diaspora Yuli Edelstein stated that the settlement freeze should be lifted and construction resumed, as \"It's our land anyway!\"\n", + "- Construction of new settlement units in Har Homa, just days after Netanyahu announced the freeze in November 2009, shows that Netanyahu authorized the building of new settlement units.\n", + "This email discusses a leaked poll regarding American perceptions of the Israeli operation against the Gaza Flotilla. The poll found that:\n", + "- 56% of Americans agree there is a humanitarian crisis in Gaza.\n", + "- 43% of Americans agree that people in Gaza are starving.\n", + "- 34% of Americans support the Israeli operation against the Flotilla.\n", + "- 20% of Americans felt support for Israel following the announcement of easing the Gaza closure.\n", + "**Email Summary Related to Israel:**\n", + "\n", + "**Subject:** SECURITY COUNCIL PRESIDENTIAL STATEMENT ON THE RECENT INCIDENT IN THE EASTERN MEDITERRANEAN\n", + "\n", + "**Summary:**\n", + "\n", + "* The Security Council condemns Israel's military actions against the Gaza-bound convoy.\n", + "* The Council demands immediate release of the ships and civilians detained by Israel.\n", + "* The Council urges Israel to allow consular access, repatriation of casualties, and delivery of humanitarian aid.\n", + "* The Council calls for a prompt and impartial investigation into the incident.\n", + "* The Council stresses the need to alleviate the humanitarian crisis in Gaza and facilitate regular movement of goods and people.\n", + "* The Council emphasizes the importance of a two-state solution to the Israeli-Palestinian conflict.\n", + "* The Council urges restraint and cooperation between the parties and international partners.\n", + "- Israel disputes the Obama administration's claim that the US warned Israeli officials to exercise caution and restraint while intercepting the flotilla.\n", + "- Israel's Ambassador to the US, Michael B. Oren, has denied receiving any such contact from the US administration and is seeking clarification.\n", + "There is no mention of Israel or Israeli in the email you provided.\n", + "I'm sorry but this email does not contain information related to Israel.\n", + "The provided email does not mention anything related to Israel and therefore I cannot extract the requested information.\n", + "This email contains no information related to Israel.\n", + "This email does not mention anything about Israel or Israeli, so I cannot extract the requested information.\n", + "No information related to Israel or Israelis was found in the provided email.\n", + "There is no mention of Israel or Israeli affairs in the provided email.\n", + "There is no mention of Israel in the provided text.\n", + "- Request to discuss the situation in Israel and trip cancellation;\n", + "- Sender is flexible with timing;\n", + "- Phil Gordon and Jeff Feltman are monitoring the situation;\n", + "- No formal talking points available at the time of the email.\n", + "No information related to Israel was found in this email.\n", + "There is no mention of Israel or Israeli affairs in the provided email.\n", + "**Date:** Not specified\n", + "\n", + "**Subject:** Not specified\n", + "\n", + "**Summary:** This email contains details of a scheduled trilateral meeting between Secretary Clinton, Israeli Prime Minister Binyamin Netanyahu, and Palestinian National Authority President Mahmoud Abbas at 10:00 am, followed by a bilateral meeting at 11:30 am.\n", + "This email does not mention Israel or Israeli and is therefore not relevant to your query.\n", + "**Summary:**\n", + "\n", + "The email discusses the cancellation of a joint interview with Israeli and Palestinian media outlets. The sender objects to the cancellation, citing the following concerns:\n", + "\n", + "* High expectations for the interview, and cancellation would be perceived negatively.\n", + "* Cancellation would contribute to a negative narrative in Israeli media.\n", + "* Potential for negative symbolism by canceling on the Israeli media outlet, Channel 2.\n", + "\n", + "The sender suggests offering Channel 2 an exclusive interview in Jerusalem in two weeks as a way to mitigate the consequences of cancellation.\n", + "The context provided does not mention anything about Israel, so I cannot extract the requested information.\n", + "- 9:15 am: Joint taped interview with Udi Segal, Israeli Channel 2\n", + "No mention of Israel or Israeli in the provided email.\n", + "There is no information about Israel in the provided email.\n", + "There is no mention of Israel or Israeli in the provided email content.\n", + "- Anne-Marie Slaughter has attached a paper by Louise Diamond, an expert on psychology and negotiations who has done a lot of work between Israel and the Palestinians.\n", + "- Slaughter recommends watching a video produced by a Rand study called The Arc, which proposes a high-speed road/rail project connecting the key cities in the West Bank and Gaza.\n", + "There is no mention of Israel in the provided email.\n", + "No information related to Israel is present in this email.\n", + "I am unable to provide a summary since the email you sent does not mention Israel.\n", + "No information related to Israel or Israeli was found in the provided email.\n", + "This email does not relate to Israel.\n", + "There is no mention of Israel in the provided email.\n", + "This email is not related to Israel or Israeli.\n", + "No information relevant to Israel or Israeli is found in this email.\n", + "**Summary:**\n", + "\n", + "* Israel Prime Minister Netanyahu is in a strong political position with a stable ruling coalition.\n", + "* Netanyahu could have extended the settlement freeze before its expiry to support US peacemaking efforts, but chose not to.\n", + "* Netanyahu is concerned about keeping Lieberman in the government to prevent him from garnering support from the right-wing.\n", + "* Kadima leader Tzipi Livni is willing to bring her party into the government without demanding rotation if Netanyahu is serious about negotiating peace.\n", + "* Netanyahu's negotiating tactics are self-defeating and contribute to distrust on the Palestinian side.\n", + "* Israeli public is ready for a peace deal; failure to make a serious move will further delegitimize Israel internationally.\n", + "* Yuval Diskin, former head of Shin Bet, warns that Palestinian security motivation may disappear if Netanyahu does not quickly demonstrate seriousness to Abu Mazen.\n", + "The email does not mention Israel or any Israeli figures.\n", + "- Jake suggests a condolence call to Netanyahu and offering assistance.\n", + "- About forty people died due to a forest fire in northern Israel.\n", + "No information related to Israel is present in the provided email.\n", + "9:00 am Secretary's Conference Room\n", + "MEETING w/ISRAELI CHIEF NEGOTIATOR YITZHAK MOLHO\n", + "\n" + ] + } + ], + "source": [ + "file_path = \"results_email_summary.txt\"\n", + "\n", + "# Open the file and read its content\n", + "with open(file_path, \"r\") as file:\n", + " provided_data = file.read()\n", + "\n", + "# Display the content\n", + "print(provided_data)" + ] + }, { "cell_type": "markdown", "metadata": {}, @@ -50,7 +228,7 @@ }, { "cell_type": "code", - "execution_count": 15, + "execution_count": 3, "metadata": {}, "outputs": [], "source": [ @@ -60,6 +238,7 @@ "# !pip -q install langchain_experimental langchain_core\n", "# !pip install --upgrade langchain\n", "\n", + "import os\n", "import google.generativeai as genai\n", "from IPython.display import display\n", "from IPython.display import Markdown\n", @@ -82,9 +261,21 @@ }, { "cell_type": "code", - "execution_count": 16, + "execution_count": 4, "metadata": {}, - "outputs": [], + "outputs": [ + { + "ename": "NameError", + "evalue": "name 'os' is not defined", + "output_type": "error", + "traceback": [ + "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[1;31mNameError\u001b[0m Traceback (most recent call last)", + "Cell \u001b[1;32mIn[4], line 6\u001b[0m\n\u001b[0;32m 3\u001b[0m load_dotenv(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mmy_config.env\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n\u001b[0;32m 5\u001b[0m \u001b[38;5;66;03m# Access the environment variables\u001b[39;00m\n\u001b[1;32m----> 6\u001b[0m GOOGLE_AI_STUDIO \u001b[38;5;241m=\u001b[39m \u001b[43mos\u001b[49m\u001b[38;5;241m.\u001b[39mgetenv(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mGOOGLE_AI_STUDIO2\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n\u001b[0;32m 8\u001b[0m \u001b[38;5;66;03m# replace your own Gemini API key\u001b[39;00m\n\u001b[0;32m 9\u001b[0m genai\u001b[38;5;241m.\u001b[39mconfigure(api_key\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mGOOGLE_AI_STUDIO\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n", + "\u001b[1;31mNameError\u001b[0m: name 'os' is not defined" + ] + } + ], "source": [ "# ================ Key configuration===========\n", "# Load environment variables from the .env file\n", @@ -126,7 +317,7 @@ }, { "cell_type": "code", - "execution_count": 18, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -185,48 +376,50 @@ }, { "cell_type": "code", - "execution_count": 13, + "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/markdown": [ - "**Political Insights Based on Leaked Hillary Clinton Emails Related to Israel**\n", + "**Political Insights from Leaked Hillary Clinton Emails on Israel:**\n", "\n", - "The leaked email summaries provide valuable insights into the political dynamics surrounding Israel during Hillary Clinton's tenure as Secretary of State. These insights can be categorized as follows:\n", + "**1. Israel's Settlement Policy and US Concerns:**\n", "\n", - "**1. Diplomatic Challenges and Negotiations:**\n", + "* Emails reveal ongoing concerns within the US administration regarding Israel's settlement activities in the West Bank, particularly during the Obama administration's push for a two-state solution.\n", + "* US officials expressed concerns that settlements undermined the viability of a Palestinian state and hindered peace negotiations.\n", + "* Despite US pressure, Israel continued settlement construction, leading to tensions between the two countries.\n", "\n", - "* The emails reveal ongoing diplomatic efforts to facilitate peace talks between Israel and the Palestinians, highlighting the complexities and challenges involved in negotiations.\n", - "* They shed light on the delicate balance between maintaining good relations with Israel while also addressing concerns from Arab and Palestinian partners.\n", + "**2. US-Israel Relations and the Gaza Flotilla Incident:**\n", "\n", - "**2. Settlement Freeze and Construction:**\n", + "* Emails shed light on the strained relations between the US and Israel following the Israeli military's interception of the Gaza-bound flotilla in 2010.\n", + "* US officials condemned Israel's actions and demanded an investigation, while Israel denied receiving warnings from the US to exercise restraint.\n", + "* The incident highlighted the challenges in maintaining a close alliance amidst differing perspectives on security and humanitarian concerns.\n", "\n", - "* The emails discuss the controversial issue of Israeli settlement construction in the West Bank, including the Obama administration's efforts to secure a settlement freeze and Israel's reluctance to fully comply.\n", - "* They provide evidence of ongoing tensions between the US and Israel over this issue, which remains a significant obstacle to peace efforts.\n", + "**3. Netanyahu's Political Calculus and Peace Negotiations:**\n", "\n", - "**3. Public Perception and International Pressure:**\n", + "* Emails provide insights into Israeli Prime Minister Benjamin Netanyahu's political considerations and his approach to peace negotiations.\n", + "* Netanyahu faced pressure from both within his coalition and the Israeli public to maintain a strong stance on settlements and security.\n", + "* Emails suggest that Netanyahu's reluctance to extend the settlement freeze and his negotiating tactics contributed to distrust on the Palestinian side.\n", "\n", - "* The emails reflect the challenges faced by Israel in managing its international image, particularly in the wake of incidents like the Gaza Flotilla raid.\n", - "* They show how the US administration attempted to mediate between Israel and the international community, emphasizing the importance of accountability and restraint.\n", + "**4. International Pressure on Israel:**\n", "\n", - "**4. Domestic Political Considerations:**\n", + "* Emails indicate that Israel faced growing international pressure over its settlement policy and its handling of the Gaza conflict.\n", + "* The UN Security Council issued a statement condemning Israel's actions against the flotilla, highlighting the international community's concerns.\n", + "* Israel's actions also led to a decline in support for Israel among the American public.\n", "\n", - "* The emails provide glimpses into the domestic political dynamics within Israel, including the influence of right-wing parties and the challenges faced by Prime Minister Netanyahu in balancing their demands with international pressure.\n", - "* They highlight the complexities of Israeli coalition politics and the impact on decision-making.\n", + "**5. Domestic Challenges and Public Opinion:**\n", "\n", - "**5. Security Concerns:**\n", + "* Emails reveal that Netanyahu's government faced domestic challenges, including concerns about keeping right-wing parties in the coalition and managing public expectations.\n", + "* Israeli public opinion was reportedly ready for a peace deal, but Netanyahu's failure to make a serious move risked delegitimizing Israel internationally.\n", "\n", - "* The emails touch on security-related issues, such as the humanitarian crisis in Gaza and the need for a two-state solution to address both Israeli security concerns and Palestinian aspirations.\n", - "* They demonstrate the interconnectedness of political and security matters in the region.\n", - "\n", - "Overall, these leaked emails offer valuable insights into the complexities of US-Israel relations, the challenges of peace negotiations, and the political dynamics shaping Israel's domestic and foreign policy. They underscore the importance of diplomacy, dialogue, and a balanced approach to address the multifaceted issues in the region." + "**Overall, the leaked emails provide valuable insights into the complexities of US-Israel relations, the challenges of peace negotiations, and the political dynamics within Israel.**" ], "text/plain": [ "" ] }, - "execution_count": 13, + "execution_count": 22, "metadata": {}, "output_type": "execute_result" } @@ -236,42 +429,27 @@ "output_parser = StrOutputParser()\n", "\n", "# This line creates a processing chain using the pipe (|) operator.\n", + "\n", "chain = prompt | model | output_parser\n", "\n", - "with open(r\".\\role_political_analyst.txt\", \"r\") as file:\n", - " role = file.read()\n", + "role = \"I want you to act as a political analyst specializing in Israel. I will present you with various political scenarios, historical events, or current affairs related to Israel, and your task is to analyze them from a political, social, and economic perspective. Provide insights into the implications of these scenarios, how they reflect on Israel's domestic and foreign policy, and what potential outcomes or future developments could arise from them. Remember, your responses should be based on factual analysis, taking into account Israel's political landscape, its relationships with neighboring countries, and the broader geopolitical context. Do not provide personal opinions or speculative predictions that cannot be supported by current facts or historical precedent.\"\n", "\n", - "with open(r\".\\results_email_summary.txt\", \"r\") as file:\n", - " provided_data = file.read()\n", - "\n", - "with open(r\".\\start_political_analyst.txt\", \"r\") as file:\n", - " start = file.read()\n", + "start = \"My first scenario involves analyzing a collection of leaked email summaries obtained from Hillary Clinton's private email server. These emails are specifically related to Israel. Describe political insights based on these emails.\"\n", "\n", "result = chain.invoke(\n", - "\n", - "\n", " {\n", - "\n", - "\n", " \"role\": role,\n", - "\n", - "\n", " \"provided_data\": provided_data,\n", - "\n", - "\n", " \"start\": start,\n", - "\n", - "\n", " }\n", ")\n", "\n", - "\n", "Markdown(result)" ] }, { "cell_type": "code", - "execution_count": 5, + "execution_count": null, "metadata": {}, "outputs": [ {