digital-forensics-labs/digital-forensics-lab
1
0
Fork 0
mirror of https://github.com/frankwxu/digital-forensics-lab.git synced 2026-02-21 11:17:52 +00:00
Code Issues Packages Projects Releases Wiki Activity

add colab urls for each CIKM2024 labs

Browse Source
This commit is contained in:
Frank Xu
2024-07-27 10:38:36 -04:00
parent bb60cd7984
commit 265b9fa024
3 changed files with 79 additions and 79 deletions
Download Patch File Download Diff File Expand all files Collapse all files

68
AI4Forensics/CKIM2024/BrowserHistory/Eric/profile_browser_history_Eric.ipynb
View File

@@ -26,7 +26,7 @@
" \n",
"- source: [Google takeout](https://takeout.google.com/settings/takeout?pli=1)\n",
" - trimmed to 113 records\n",
" - sample record shown as follows. ONLY **title** is used for profiling in this demo\n",
" - sample record shown as follows. ONLY **title** and **timestamp** are used for profiling in this demo\n",
"```\n",
" {\n",
" \"favicon_url\": \"https://leetcode.com/favicon.ico\",\n",
@@ -59,7 +59,7 @@
},
{
"cell_type": "code",
"execution_count": 20,
"execution_count": 1,
"metadata": {},
"outputs": [
{
@@ -208,7 +208,7 @@
},
{
"cell_type": "code",
"execution_count": 21,
"execution_count": 2,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
@@ -218,11 +218,11 @@
},
"outputs": [],
"source": [
"# !pip -q install google-generativeai\n",
"# !pip -q install langchain-google-genai\n",
"# !pip install python-dotenv\n",
"# !pip -q install langchain_experimental langchain_core\n",
"# !pip install --upgrade langchain\n",
"#!pip -q install google-generativeai\n",
"#!pip -q install langchain-google-genai\n",
"#!pip install python-dotenv\n",
"#!pip -q install langchain_experimental langchain_core\n",
"#!pip install --upgrade langchain\n",
"\n",
"import os\n",
"import google.generativeai as genai\n",
@@ -247,12 +247,12 @@
"#### Step 2: Config LangChain with gemini\n",
"- You `MUST` have a Gemini key\n",
"- You can load an an api key from `api_key.txt` file\n",
"- or, hard code your open api key"
"- or, hard code your open api later when you create a model"
]
},
{
"cell_type": "code",
"execution_count": 22,
"execution_count": 3,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
@@ -268,10 +268,7 @@
"\n",
"# Access the environment variables\n",
"GOOGLE_AI_STUDIO = os.getenv(\"GOOGLE_AI_STUDIO2\")\n",
"\n",
"# replace your own Gemini API key\n",
"genai.configure(api_key=\"GOOGLE_AI_STUDIO\")\n",
"\n",
"genai.configure(api_key=GOOGLE_AI_STUDIO)\n",
"\n",
"# ======= Gerneration configuration===========\n",
"# Set up the model\n",
@@ -297,12 +294,14 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"### Step 3: build a Gemini model with configurations"
"### Step 3: build a Gemini model with configurations\n",
"\n",
"Note: we can hard code the Gemini key here"
]
},
{
"cell_type": "code",
"execution_count": 23,
"execution_count": 4,
"metadata": {
"id": "OyqoVaYpvlcC"
},
@@ -312,7 +311,8 @@
" model=\"gemini-pro\",\n",
" generation_config=generation_config,\n",
" safety_settings=safety_settings,\n",
" google_api_key=GOOGLE_AI_STUDIO\n",
" # You can hardcode the key AIzaSyCDqn8xVJ4cFeiXSvhPUcnR60jfBLj5dO4k\n",
" google_api_key=GOOGLE_AI_STUDIO,\n",
")"
]
},
@@ -342,7 +342,7 @@
},
{
"cell_type": "code",
"execution_count": 24,
"execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
@@ -370,7 +370,7 @@
},
{
"cell_type": "code",
"execution_count": 25,
"execution_count": 6,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
@@ -385,25 +385,33 @@
"text/markdown": [
"**Psychological Profile:**\n",
"\n",
"The individual exhibits a wide range of interests, including technology, gaming, programming, home automation, sustainability, art, photography, music, and writing. This suggests a curious and multifaceted personality with a thirst for knowledge and creative expression.\n",
"The browsing history suggests that the suspect is a young adult male, likely in the 20-29 age range. They have a strong interest in technology and gadgets, particularly in gaming and software development. They also enjoy entertainment, especially video games and online content. Their interest in online forums and social media indicates a desire for social interaction and community involvement. The presence of searches related to health and wellness, education, and learning suggests a focus on self-improvement and personal growth.\n",
"\n",
"The browsing history indicates a focus on specific games and apps (e.g., Valorant, RAID: Shadow Legends, Overwatch) and technical topics (e.g., LaTeX, VR training, linked list algorithms). This suggests a methodical and goal-oriented approach to learning and problem-solving.\n",
"**Interests:**\n",
"\n",
"The repeated visits to websites related to home automation, sustainability, and smart homes imply a concern for efficiency, comfort, and the environment. This could indicate a practical and environmentally conscious mindset.\n",
"* Technology and Gadgets\n",
"* Entertainment (video games, anime, memes)\n",
"* Sports and Fitness (Overwatch)\n",
"* Hobbies and Crafts (online game strategy optimization)\n",
"* Health and Wellness\n",
"* Education and Learning (programming, data structures)\n",
"* Socializing and Community (Reddit, Discord)\n",
"\n",
"The browsing history also includes visits to social media platforms (e.g., Reddit, Twitter), online forums, and community-based websites. This suggests a need for social connection and a desire to engage with others who share similar interests.\n",
"**Location:**\n",
"\n",
"**Age Range:** 20-29\n",
"The browsing history does not provide specific clues about the suspect's location within the United States. However, given the prevalence of gaming, online content consumption, and social media usage, it is likely that the suspect resides in an urban or suburban area with good internet access.\n",
"\n",
"**Interests:** Technology and Gadgets, Entertainment, Hobbies and Crafts, Education and Learning, Socializing and Community\n",
"**Additional Observations:**\n",
"\n",
"**Location:** California"
"* There is a pattern of late-night and early morning browsing behavior, which could indicate irregular sleep patterns or a nocturnal lifestyle.\n",
"* The suspect appears to be interested in optimizing their performance in online games, suggesting a competitive or achievement-oriented mindset.\n",
"* The presence of searches related to memes and humorous content indicates a playful and lighthearted sense of humor."
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"execution_count": 25,
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
@@ -438,10 +446,10 @@
},
"source": [
"### Part 6: Evaluation\n",
"- Evaluating based on the age, range, and interets\n",
"- Age: We will give 1 score if the ranges are the same. If the range is off by one (i,e if the age is 30-49 and the prediction is 50-59 or 20-29), we will give 0.5 score\n",
"- Evaluating based on the age, range, and interests\n",
"- Age: We will give 1 score if the ranges are the same. If the range is off by one (i.e., if the age is 30-49 and the prediction is 50-59 or 20-29), we will give 0.5 score\n",
"- Interests: Score is the number of correct interests over the largest number of interests guessed. If 4 interests are predicted and 3 are correct, it will be given a score of 3/4 or 0.75\n",
"- Location: Score of 1 if prediction is correct. Score of 0.6 if the state borders the state. 0.3 if they are the same section of the United states. Sections are divided into Northeast, Southeast, Midwest, Southwest, and West\n"
"- Location: Score of 1 if prediction is correct. Score of 0.6 if the state borders the state. 0.3 if they are the same section of the United States. Sections are divided into Northeast, Southeast, Midwest, Southwest, and West\n"
]
},
{

78
AI4Forensics/CKIM2024/HillaryEmails/email_analysis_political_insight.ipynb
View File

@@ -263,19 +263,7 @@
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"ename": "NameError",
"evalue": "name 'os' is not defined",
"output_type": "error",
"traceback": [
"\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[1;31mNameError\u001b[0m Traceback (most recent call last)",
"Cell \u001b[1;32mIn[4], line 6\u001b[0m\n\u001b[0;32m 3\u001b[0m load_dotenv(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mmy_config.env\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n\u001b[0;32m 5\u001b[0m \u001b[38;5;66;03m# Access the environment variables\u001b[39;00m\n\u001b[1;32m----> 6\u001b[0m GOOGLE_AI_STUDIO \u001b[38;5;241m=\u001b[39m \u001b[43mos\u001b[49m\u001b[38;5;241m.\u001b[39mgetenv(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mGOOGLE_AI_STUDIO2\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n\u001b[0;32m 8\u001b[0m \u001b[38;5;66;03m# replace your own Gemini API key\u001b[39;00m\n\u001b[0;32m 9\u001b[0m genai\u001b[38;5;241m.\u001b[39mconfigure(api_key\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mGOOGLE_AI_STUDIO\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n",
"\u001b[1;31mNameError\u001b[0m: name 'os' is not defined"
]
}
],
"outputs": [],
"source": [
"# ================ Key configuration===========\n",
"# Load environment variables from the .env file\n",
@@ -283,10 +271,7 @@
"\n",
"# Access the environment variables\n",
"GOOGLE_AI_STUDIO = os.getenv(\"GOOGLE_AI_STUDIO2\")\n",
"\n",
"# replace your own Gemini API key\n",
"genai.configure(api_key=\"GOOGLE_AI_STUDIO\")\n",
"\n",
"genai.configure(api_key=GOOGLE_AI_STUDIO)\n",
"\n",
"# ======= Gerneration configuration===========\n",
"# Set up the model\n",
@@ -317,7 +302,7 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
@@ -325,6 +310,7 @@
" model=\"gemini-pro\",\n",
" generation_config=generation_config,\n",
" safety_settings=safety_settings,\n",
" # You can hardcode the key AIzaSyCDqn8xVJ4cFeiXSvhPUcnR60jfBLj5dO4k\n",
" google_api_key=GOOGLE_AI_STUDIO,\n",
")"
]
@@ -344,13 +330,13 @@
"```\n",
"- `{role}, {provided_data}, and {start}` are placeholders that will be filled in later.\n",
" - `{role}`: definition specifies the role's name, overall objective, task specific context, and any applicable constraints. \n",
" - `{provided_data}`: outlines the required datasets for task completion\n",
" - `{provided_data}`: outlines the required datasets for task completion\n",
" - `{start}`: the initiation instruction serves as a trigger, prompting the role to carry out the task"
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 6,
"metadata": {},
"outputs": [],
"source": [
@@ -376,7 +362,7 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 7,
"metadata": {},
"outputs": [
{
@@ -384,42 +370,48 @@
"text/markdown": [
"**Political Insights from Leaked Hillary Clinton Emails on Israel:**\n",
"\n",
"**1. Israel's Settlement Policy and US Concerns:**\n",
"**1. US-Israel Diplomatic Tensions:**\n",
"\n",
"* Emails reveal ongoing concerns within the US administration regarding Israel's settlement activities in the West Bank, particularly during the Obama administration's push for a two-state solution.\n",
"* US officials expressed concerns that settlements undermined the viability of a Palestinian state and hindered peace negotiations.\n",
"* Despite US pressure, Israel continued settlement construction, leading to tensions between the two countries.\n",
"* Several emails reveal tensions between the US and Israel, particularly regarding Israel's settlement expansion and military actions.\n",
"* Israel's Ambassador to the US, Michael Oren, expressed concerns about the US reprimanding Israel over its conduct during Vice President Biden's visit.\n",
"* Israel disputed the Obama administration's claim that it had warned Israeli officials to exercise caution during the Gaza Flotilla interception.\n",
"\n",
"**2. US-Israel Relations and the Gaza Flotilla Incident:**\n",
"**2. Internal Israeli Politics:**\n",
"\n",
"* Emails shed light on the strained relations between the US and Israel following the Israeli military's interception of the Gaza-bound flotilla in 2010.\n",
"* US officials condemned Israel's actions and demanded an investigation, while Israel denied receiving warnings from the US to exercise restraint.\n",
"* The incident highlighted the challenges in maintaining a close alliance amidst differing perspectives on security and humanitarian concerns.\n",
"* Emails discuss the political dynamics within Israel's ruling coalition, including the pressure on Prime Minister Netanyahu from right-wing parties to block the renewal of the settlement freeze.\n",
"* Netanyahu's motivations for not extending the settlement freeze are analyzed, suggesting concerns about keeping coalition partners satisfied.\n",
"* The potential impact of Kadima leader Tzipi Livni's willingness to join the government without demanding rotation is examined.\n",
"\n",
"**3. Netanyahu's Political Calculus and Peace Negotiations:**\n",
"**3. Peace Negotiations and International Pressure:**\n",
"\n",
"* Emails provide insights into Israeli Prime Minister Benjamin Netanyahu's political considerations and his approach to peace negotiations.\n",
"* Netanyahu faced pressure from both within his coalition and the Israeli public to maintain a strong stance on settlements and security.\n",
"* Emails suggest that Netanyahu's reluctance to extend the settlement freeze and his negotiating tactics contributed to distrust on the Palestinian side.\n",
"* Emails highlight the US administration's efforts to facilitate peace negotiations between Israel and the Palestinians.\n",
"* The international community's concerns about Israel's settlement policy and its impact on peace prospects are evident.\n",
"* Former Shin Bet chief Yuval Diskin's warning about the potential erosion of Palestinian security motivation if Netanyahu does not demonstrate seriousness about peace is noteworthy.\n",
"\n",
"**4. International Pressure on Israel:**\n",
"**4. Public Opinion and Political Strategy:**\n",
"\n",
"* Emails indicate that Israel faced growing international pressure over its settlement policy and its handling of the Gaza conflict.\n",
"* The UN Security Council issued a statement condemning Israel's actions against the flotilla, highlighting the international community's concerns.\n",
"* Israel's actions also led to a decline in support for Israel among the American public.\n",
"* Polls discussed in the emails indicate American support for a two-state solution and concern about the humanitarian crisis in Gaza.\n",
"* Netanyahu's negotiating tactics are scrutinized, with analysts suggesting they contribute to distrust on the Palestinian side.\n",
"* The Israeli public's readiness for a peace deal is highlighted, with warnings that failure to make a serious move could further delegitimize Israel internationally.\n",
"\n",
"**5. Domestic Challenges and Public Opinion:**\n",
"**Social and Economic Implications:**\n",
"\n",
"* Emails reveal that Netanyahu's government faced domestic challenges, including concerns about keeping right-wing parties in the coalition and managing public expectations.\n",
"* Israeli public opinion was reportedly ready for a peace deal, but Netanyahu's failure to make a serious move risked delegitimizing Israel internationally.\n",
"* The emails do not provide significant insights into the social or economic implications of the discussed political issues. However, the humanitarian crisis in Gaza and the potential impact of settlement expansion on the Palestinian economy are mentioned in passing.\n",
"\n",
"**Overall, the leaked emails provide valuable insights into the complexities of US-Israel relations, the challenges of peace negotiations, and the political dynamics within Israel.**"
"**Potential Future Developments:**\n",
"\n",
"Based on the analysis of these emails, potential future developments could include:\n",
"\n",
"* Continued tensions between the US and Israel over settlement policy and other issues.\n",
"* Further political instability within Israel's coalition government.\n",
"* Stalled peace negotiations and international pressure on Israel.\n",
"* Growing public dissatisfaction in Israel if Netanyahu fails to make progress towards peace."
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"execution_count": 22,
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
@@ -449,7 +441,7 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 8,
"metadata": {},
"outputs": [
{

12
AI4Forensics/CKIM2024/readme.md
View File

@@ -31,13 +31,13 @@ By fostering a collaborative learning environment, this tutorial aims to empower
- Introduction
- [Forensic evidence entity recognition (hands-on lab)](#forensic-evidence-analysis)
- [Evidence entity recognition](PhishingAttack/PhishingAttackScenarioDemo/01_evidence_entity_recognition.ipynb)
- [Visualize evidence and their relations](PhishingAttack/PhishingAttackScenarioDemo/02_evidence_knowledge_dot_generator.ipynb)
- [Evidence entity recognition](https://colab.research.google.com/github/frankwxu/digital-forensics-lab/blob/main/AI4Forensics/CKIM2024/PhishingAttack/PhishingAttackScenarioDemo/01_evidence_entity_recognition.ipynb)
- [Visualize evidence and their relations](https://colab.research.google.com/github/frankwxu/digital-forensics-lab/blob/main/AI4Forensics/CKIM2024/PhishingAttack/PhishingAttackScenarioDemo/02_evidence_knowledge_dot_generator.ipynb)
- [Evidence knowledge graphs reconstruction (hands-on lab)](#forensic-evidence-analysis)
- [Construct a knowledge graph in STIX (zero-shot)](PhishingAttack/PhishingAttackScenarioDemo/03_evidence_stix_zeroshot.ipynb)
- [Construct a knowledge graph in STIX (one-shot)](PhishingAttack/PhishingAttackScenarioDemo/04_evidence_stix_oneshot.ipynb)
- [Compare one-shot vs. zero-shot](PhishingAttack/PhishingAttackScenarioDemo/05_evidence_stix_dot_generator.ipynb)
- [Profiling suspect based on browser history (hands-on lab)](BrowserHistory/Eric/profile_browser_history_Eric.ipynb)
- [Construct a knowledge graph in STIX (zero-shot)](https://colab.research.google.com/github/frankwxu/digital-forensics-lab/blob/main/AI4Forensics/CKIM2024/PhishingAttack/PhishingAttackScenarioDemo/03_evidence_stix_zeroshot.ipynb)
- [Construct a knowledge graph in STIX (one-shot)](https://colab.research.google.com/github/frankwxu/digital-forensics-lab/blob/main/AI4Forensics/CKIM2024/PhishingAttack/PhishingAttackScenarioDemo/04_evidence_stix_oneshot.ipynb)
- [Compare one-shot vs. zero-shot](https://colab.research.google.com/github/frankwxu/digital-forensics-lab/blob/main/AI4Forensics/CKIM2024/PhishingAttack/PhishingAttackScenarioDemo/05_evidence_stix_dot_generator.ipynb)
- [Profiling suspect based on browser history (hands-on lab)](https://colab.research.google.com/github/frankwxu/digital-forensics-lab/blob/main/AI4Forensics/CKIM2024/BrowserHistory/Eric/profile_browser_history_Eric.ipynb)
- [Political insights analysis based on Hillary's leaked Emails (hands-on lab)](#political-insight-analysis-leveraging-llms)
- Challenges and Limitations of Leveraging LLM in Digital Forensics
- Conclusion