Skip to content

Friendzone Lite - AI Memory Capture System#

Overview#

Friendzone Lite is an AI-powered memory capture and organization system that helps you extract, refine, and structure memories from images through intelligent conversation. It transforms images into structured, searchable memories through natural language interaction.

Features#

Image-Based Memory Extraction#

  • Upload images via URL to start memory capture sessions
  • AI analyzes visual content to understand the context
  • Extracts initial memory details from image content

Interactive Memory Refinement#

  • Conversational interface to gather missing details
  • AI asks targeted follow-up questions
  • Progressive refinement of memory information

Structured Memory Organization#

  • When: Date information (YYYY-MM-DD format)
  • Who: People involved in the memory
  • Where: Locations relevant to the memory
  • What: Description of what the memory is about

Completion Tracking#

  • Real-time progress indication
  • Summary generation when details are complete
  • Clear completion status with final memory summary

Architecture#

Core Components#

  • Memory Processing Function: update_memory_details() - AI-powered memory extraction and refinement
  • Session Node: Maintains persistent memory state across interactions
  • Update Walker: Handles user interactions and memory updates
obj Response {
    has follow_up_questions: str;    # Next question to ask
    has summary: str;                # Concise memory summary
    has when: str;                   # Date in YYYY-MM-DD format
    has who: List[str];              # Names of people involved
    has what: str;                   # What the memory is about
    has where: List[str];            # Relevant locations
    has terminate_conversation: bool; # Completion flag
    has show_summary: bool;          # Display summary flag
}

Complete Code Preview#

Here's what you'll build - an AI memory capture system in just two files:

Friendzone Lite Frontend

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
import from byllm.llm { Model }
import from byllm.types { Image }
import from typing { List, Dict, Any }
import os;

glob llm = Model(model_name="gemini/gemini-2.5-flash");

obj Response {
    has follow_up_questions: str,
        summary: str,
        when: str,
        who: List[str],
        what: str,
        where: List[str],
        terminate_conversation: bool,
        show_summary: bool;
}

sem Response= "Memory details refined using the user's input and prior context.";
sem Response. follow_up_questions= "Ask one follow-up question to continue the conversation. If all required details are present, ask to terminate the conversation.";
sem Response. summary= "A concise summary of the memory.";
sem Response. when= "The date of the memory in YYYY-MM-DD format.";
sem Response. who= "Exact names of people in the memory (e.g., [John, Kate]); return [] if none.";
sem Response. where= "List of places relevant to the memory.";
sem Response. what= "What the memory is about.";
sem Response. show_summary= "True if all required details are present and the summary should be shown.";
sem Response. terminate_conversation= "True if the user asked to terminate the conversation; otherwise false.";

""" Update and extract memory details based on user input and context."""
def update_memory_details(
    image: Image,
    utterance: str = "",
    summary: str = "",
    when: str = "",
    who: List[str] = [],
    where: List[str] = [],
    what: str = "",
    conversation: List[dict] = [],
    show_summary: bool = False,
    terminate_conversation: bool = False
) -> Response by llm();

node session {
    has summary: str = "",
        when: str = "",
        who: list = [],
        where: list = [],
        what: str = "",
        conversation: list = [],
        image_url: str = "",
        show_summary: bool = False,
        terminate_conversation: bool = False;
}

walker update_session {
    has image_url: str,
        utterance: str = "";

    can visit_session with `root entry {
        # Find or create session for this image
        matching_sessions = [
            sess
            for sess in [-->](`?session)
            if sess.image_url == self.image_url and not sess.terminate_conversation
        ];

        if matching_sessions {
            visit matching_sessions[0];
        } else {
            session_node = here ++> session();
            visit session_node[0];
        }
    }

    can update_session with session entry {
        if here.image_url == "" {
            here.image_url = self.image_url;
        }
        response = update_memory_details(
            image=Image(url=self.image_url),
            utterance=self.utterance,
            summary=here.summary,
            when=here.when,
            who=here.who,
            where=here.where,
            what=here.what,
            conversation=here.conversation,
            show_summary=here.show_summary,
            terminate_conversation=here.terminate_conversation
        );
        here.summary = response.summary;
        here.when = response.when;
        here.who = response.who;
        here.where = response.where;
        here.what = response.what;
        here.show_summary = response.show_summary;
        self.show_summary = response.show_summary;
        here.terminate_conversation = response.terminate_conversation;
        self.terminate_conversation = response.terminate_conversation;
        here.conversation = here.conversation + [
            {"role": "user", "content": self.utterance}
        ] + [{"role": "assistant", "content": response.follow_up_questions}];

        if response.show_summary {
            if response.terminate_conversation {
                print("Assistant: " + response.follow_up_questions);
                print("Final Memory Summary:" + response.summary);
            } else {
                print("Memory Summary (in progress):" + response.summary);
                print("Assistant: " + response.follow_up_questions);
            }
        } else {
            print("Assistant: " + response.follow_up_questions);
        }

        # Report session state for frontend
        report {
            "summary": here.summary,
            "when": here.when,
            "who": here.who,
            "where": here.where,
            "what": here.what,
            "show_summary": here.show_summary,
            "terminate_conversation": here.terminate_conversation,
            "follow_up_questions": response.follow_up_questions,
            "conversation": here.conversation,
            "image_url": here.image_url
        } ;
    }
}

walker get_memories {
    can get_all_memories with `root entry {
        memories = [];
        for mem in [-->](`?session) {
            # Only include completed memories
            if mem.terminate_conversation and mem.show_summary {
                memories.append(
                    {
                        "summary": mem.summary,
                        "when": mem.when,
                        "who": mem.who,
                        "where": mem.where,
                        "what": mem.what,
                        "image_url": mem.image_url
                    }
                );
            }
        }
        report {"memories": memories} ;
    }
}

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
import streamlit as st;
import requests;

def bootstrap_frontend(token: str) {
    st.set_page_config(
        page_title="Friendzone Lite - Memory Assistant", page_icon="🧠", layout="wide"
    );

    st.title("🧠 Friendzone Lite - Memory Assistant");
    st.markdown("✨ Capture and organize your memories from images");

    # Initialize session state
    if "memory_results" not in st.session_state {
        st.session_state.memory_results = None;
    }
    if "conversation_active" not in st.session_state {
        st.session_state.conversation_active = False;
    }
    if "image_url" not in st.session_state {
        st.session_state.image_url = "";
    }
    if "conversation_history" not in st.session_state {
        st.session_state.conversation_history = [];
    }
    if "loading" not in st.session_state {
        st.session_state.loading = False;
    }
    if "view_memories" not in st.session_state {
        st.session_state.view_memories = False;
    }
    if "captured_memories" not in st.session_state {
        st.session_state.captured_memories = [];
    }

    # Main interface
    st.markdown("### 📷 Start Your Memory Journey");

    # Add button to view captured memories
    columns = st.columns([3, 1]);
    with columns[1] {
        if st.button("📚 View Captured Memories") {
            st.session_state.view_memories = not st.session_state.view_memories;
            if st.session_state.view_memories {
                # Fetch memories from backend
                try {
                    response = requests.post(
                        "http://localhost:8000/walker/get_memories",
                        json={},
                        headers={"Authorization": f"Bearer {token}"}
                    );
                    if response.status_code == 200 {
                        result = response.json();
                        st.session_state.captured_memories = result.get(
                            "reports", [{}]
                        )[0].get(
                            "memories", []
                        );
                    }
                } except Exception as e {
                    st.error(f"❌ Error loading memories: {str(e)}");
                }
            }
            st.rerun();
        }
    }

    # Display captured memories if view_memories is True
    if st.session_state.view_memories {
        st.markdown("---");
        st.markdown("### 📚 Captured Memories");
        if st.session_state.captured_memories {
            mem_idx = 0;
            for memory in st.session_state.captured_memories {
                mem_idx = mem_idx + 1;
                with st.expander(
                    f"Memory {mem_idx}: {memory.get('summary', 'Untitled')[:50]}...",
                    expanded=False
                ) {
                    mem_columns = st.columns([1, 2]);
                    with mem_columns[0] {
                        if memory.get("image_url") {
                            try {
                                st.image(
                                    memory["image_url"],
                                    caption="Memory Image",
                                    use_column_width=True
                                );
                            } except Exception {
                                st.write("🖼️ Image not available");
                            }
                        }
                    }
                    with mem_columns[1] {
                        st.markdown(f"**📋 Summary:**");
                        st.write(memory.get("summary", "N/A"));
                        st.markdown(
                            f"**📅 When:** {memory.get('when', 'Not specified')}"
                        );
                        who_list = memory.get("who", []);
                        st.markdown(
                            f"**👥 Who:** {', '.join(who_list)
                            if who_list
                            else 'Not specified'}"
                        );
                        where_list = memory.get("where", []);
                        st.markdown(
                            f"**📍 Where:** {', '.join(where_list)
                            if where_list
                            else 'Not specified'}"
                        );
                        st.markdown(
                            f"**🎯 What:** {memory.get('what', 'Not specified')}"
                        );
                    }
                }
            }
        } else {
            st.info(
                "📭 No captured memories yet. Start a memory session to create your first memory!"
            );
        }
        st.markdown("---");
    }

    # Image URL input section
    if not st.session_state.conversation_active {
        st.markdown("#### 🌐 Enter Image URL");
        image_url = st.text_input(
            "🔗 Image URL:",
            placeholder="https://example.com/your-image.jpg",
            help="Enter the URL of an image containing a memory you'd like to capture"
        );
        if st.button("🚀 Start Memory Session") {
            if image_url.strip() {
                st.session_state.image_url = image_url;
                st.session_state.conversation_active = True;
                st.session_state.loading = True;
                st.rerun();
            } else {
                st.warning("⚠️ Please enter a valid image URL!");
            }
        }
    }

    # Active conversation section
    if st.session_state.conversation_active {
        # Display current image
        st.markdown("#### 📷 Current Image");
        try {
            st.image(
                st.session_state.image_url,
                caption="Memory Image",
                use_column_width=True
            );
        } except Exception {
            st.warning("⚠️ Could not display image. Proceeding with URL...");
        }
        # Initialize session if loading
        if st.session_state.loading and not st.session_state.memory_results {
            with st.spinner("🧠 AI is analyzing your image...") {
                try {
                    response = requests.post(
                        "http://localhost:8000/walker/update_session",
                        json={"image_url": st.session_state.image_url, "utterance": ""},
                        headers={"Authorization": f"Bearer {token}"}
                    );

                    if response.status_code == 200 {
                        result = response.json();
                        st.session_state.memory_results = result.get("reports", [{}])[
                            0
                        ];
                        st.session_state.loading = False;
                        st.success("✅ Memory session started!");
                        st.rerun();
                    } else {
                        st.error(f"❌ Failed to start session: {response.text}");
                        st.session_state.loading = False;
                    }
                } except Exception as e {
                    st.error(f"❌ Error: {str(e)}");
                    st.session_state.loading = False;
                }
            }
        }
        # Display memory progress
        if st.session_state.memory_results {
            results = st.session_state.memory_results;
            # Memory summary section
            st.markdown("#### 📝 Memory Details");
            st.markdown("**📅 When:**");
            when_val = results.get("when", "Not specified");
            if when_val {
                st.info(when_val);
            } else {
                st.info("Not specified");
            }
            st.markdown("**👥 Who:**");
            who_val = results.get("who", []);
            if who_val {
                st.info(", ".join(who_val));
            } else {
                st.info("Not specified");
            }
            st.markdown("**📍 Where:**");
            where_val = results.get("where", []);
            if where_val {
                st.info(", ".join(where_val));
            } else {
                st.info("Not specified");
            }
            st.markdown("**🎯 What:**");
            what_val = results.get("what", "");
            if what_val {
                st.info(what_val);
            } else {
                st.info("Not specified");
            }
            # Summary
            if results.get("summary") {
                st.markdown("**📋 Memory Summary:**");
                if results.get("show_summary", False) {
                    if results.get("terminate_conversation", False) {
                        st.success(f"✅ Final Memory Summary: {results['summary']}");
                    } else {
                        st.info(
                            f"📝 Memory Summary (in progress): {results['summary']}"
                        );
                    }
                } else {
                    st.write("📝 Building memory summary...");
                }
            }
            # Conversation section
            st.markdown("#### 💬 Conversation");
            # Display conversation history
            if st.session_state.conversation_history {
                for msg in st.session_state.conversation_history {
                    if msg["role"] == "user" {
                        st.chat_message("user").write(f"👤 {msg['content']}");
                    } else {
                        st.chat_message("assistant").write(f"🤖 {msg['content']}");
                    }
                }
            }
            # Check if conversation is complete
            if results.get("show_summary", False)
            and results.get("terminate_conversation", False) {
                st.success("🎉 Memory capture complete!");
                if st.button("🔄 Start New Memory Session") {
                    st.session_state.conversation_active = False;
                    st.session_state.memory_results = None;
                    st.session_state.image_url = "";
                    st.session_state.conversation_history = [];
                    st.session_state.loading = False;
                    st.rerun();
                }
            } else {
                # User input for continuing conversation
                user_input = st.chat_input("Continue the conversation...");
                if user_input {
                    # Add user message to history
                    st.session_state.conversation_history.append(
                        {"role": "user", "content": user_input}
                    );
                    # Send to backend
                    with st.spinner("🤖 AI is processing your response...") {
                        try {
                            response = requests.post(
                                "http://localhost:8000/walker/update_session",
                                json={
                                    "image_url": st.session_state.image_url,
                                    "utterance": user_input
                                },
                                headers={"Authorization": f"Bearer {token}"}
                            );

                            if response.status_code == 200 {
                                result = response.json();
                                st.session_state.memory_results = result.get(
                                    "reports", [{}]
                                )[0];
                                # Add assistant response to history
                                if "follow_up_questions" in st.session_state.memory_results {
                                    st.session_state.conversation_history.append(
                                        {
                                            "role": "assistant",
                                            "content": st.session_state.memory_results[
                                                "follow_up_questions"
                                            ]
                                        }
                                    );
                                }
                                st.rerun();
                            } else {
                                st.error(
                                    f"❌ Failed to process response: {response.text}"
                                );
                            }
                        } except Exception as e {
                            st.error(f"❌ Error: {str(e)}");
                        }
                    }
                }
            }
        }
        # Reset session button
        if st.button("🔄 Reset Session") {
            st.session_state.conversation_active = False;
            st.session_state.memory_results = None;
            st.session_state.image_url = "";
            st.session_state.conversation_history = [];
            st.session_state.loading = False;
            st.rerun();
        }
    }

    # Footer
    st.markdown("---");
    st.markdown("**🧠 Powered by Friendzone Lite AI Assistant**");
    st.markdown(
        "✨ Features: 📷 Image analysis • 💭 Memory extraction • 📝 Structured capture"
    );
}

glob INSTANCE_URL = "http://localhost:8000",
     TEST_USER_EMAIL = "test@mail.com",
     TEST_USER_PASSWORD = "password",
     response = requests.post(
         f"{INSTANCE_URL}/user/login",
         json={"email": TEST_USER_EMAIL, "password": TEST_USER_PASSWORD}
     );

with entry {
    if response.status_code != 200 {
        # Try registering the user if login fails
        response = requests.post(
            f"{INSTANCE_URL}/user/register",
            json={"email": TEST_USER_EMAIL, "password": TEST_USER_PASSWORD}
        );
        assert response.status_code == 201;
        response = requests.post(
            f"{INSTANCE_URL}/user/login",
            json={"email": TEST_USER_EMAIL, "password": TEST_USER_PASSWORD}
        );
        assert response.status_code == 200;
    }
}

glob token = response.json()["token"];

with entry {
    bootstrap_frontend(token);
}

Usage#

  1. Install dependencies:
pip install jac-streamlit requests jaclang jac-scale byllm
  1. Start the backend server:
jac start friendzone_lite.jac
  1. Run the frontend:
jac streamlit frontend.jac

Usage Workflow#

Step 1: Image Upload#

  • Provide an image URL of a photo containing a memory you want to capture
  • The system supports various image formats (JPEG, PNG, WebP)
  • Images can be from social media, cloud storage, or any accessible web location

Step 2: Initial Analysis#

The AI will:

  • Analyze the visual content of your image
  • Identify key elements like people, places, and activities
  • Generate an initial memory framework
  • Ask the first targeted question to gather more details

Step 3: Conversational Refinement#

Engage in a natural conversation where the AI will:

  • Ask specific questions about missing details
  • Focus on one aspect at a time (when, who, where, what)
  • Build upon your previous answers
  • Gradually complete the memory profile

Step 4: Memory Completion#

When sufficient details are gathered:

  • The AI will indicate the conversation is complete
  • A comprehensive summary will be generated
  • All memory details will be structured and stored

Example Memory Capture Session#

Initial Image Analysis#

User uploads: Beach vacation photo with friends

AI Initial Response: "I can see this appears to be a beach scene with several people. To help me capture this memory completely, when was this photo taken?"

Conversational Flow#

User: "This was taken last summer, around July 2024"

AI: "Great! I can see there are several people in the photo. Can you tell me who these people are?"

User: "That's me with my college friends Sarah and Mike"

AI: "Wonderful! Where was this beach vacation? Which beach or location was this at?"

User: "This was at Santa Monica Beach in California"

AI: "Perfect! What was the occasion or what were you doing during this beach visit?"

User: "We were celebrating Sarah's birthday with a beach day trip"

Final Summary#

Generated Memory:

  • When: 2024-07-15
  • Who: User, Sarah, Mike
  • Where: Santa Monica Beach, California
  • What: Beach day celebration for Sarah's birthday

Advanced Features#

Memory Categories#

The system automatically categorizes memories:

  • Social Events: Gatherings, parties, celebrations
  • Travel Experiences: Vacations, trips, explorations
  • Personal Milestones: Achievements, life events
  • Daily Moments: Casual interactions, routine activities

Context Enhancement#

  • Emotional Context: Captures feelings and atmosphere
  • Environmental Details: Weather, time of day, setting
  • Activity Recognition: Identifies specific actions and events
  • Relationship Mapping: Understands social connections

Quality Assurance#

  • Consistency Checking: Validates logical coherence of memory details
  • Completeness Assessment: Ensures all key aspects are captured
  • Accuracy Verification: Cross-references visual and textual information

Looking for the full version? This is a lite version for learning purposes. Check out the full-scale FriendZone project for a complete implementation with advanced features.

Technical Implementation#

Visual Analysis Pipeline#

  1. Image Processing: Loads and preprocesses uploaded images
  2. Object Detection: Identifies people, objects, and scenes
  3. Scene Understanding: Analyzes context and activities
  4. Initial Memory Extraction: Creates preliminary memory structure

Conversational AI System#

  1. Question Generation: Creates contextually relevant follow-up questions
  2. Response Processing: Analyzes and incorporates user answers
  3. Progress Tracking: Monitors completion of memory aspects
  4. Dialogue Management: Maintains conversation flow and coherence

Memory Storage Format#

{
  "memory_id": "unique_identifier",
  "created_at": "timestamp",
  "image_url": "source_image_url",
  "details": {
    "when": "2024-07-15",
    "who": ["User", "Sarah", "Mike"],
    "where": ["Santa Monica Beach", "California"],
    "what": "Beach day celebration for Sarah's birthday"
  },
  "summary": "Comprehensive memory description",
  "conversation_log": ["question_response_pairs"]
}

Customization Options#

Question Templates#

Modify questioning strategies for different memory types:

  • Event-focused: Emphasizes activities and occasions
  • People-centered: Prioritizes social connections and relationships
  • Location-based: Focuses on geographical and environmental details
  • Time-sensitive: Emphasizes chronological and temporal aspects

Analysis Depth#

Configure the level of detail for memory capture:

  • Quick Capture: Basic who, what, when, where
  • Detailed Analysis: Extended context and emotional elements
  • Comprehensive Profiling: Full environmental and social mapping

Integration Possibilities#

  • Photo Libraries: Connect with Google Photos, iCloud, or other services
  • Social Media: Import memories from Facebook, Instagram, or Twitter
  • Calendar Integration: Link memories with calendar events and dates
  • Search Functionality: Build searchable memory databases

Use Cases and Applications#

Personal Memory Keeping#

  • Family Archives: Document family events and milestones
  • Travel Journals: Capture vacation and trip memories
  • Life Documentation: Record personal achievements and experiences
  • Social Memories: Preserve moments with friends and loved ones

Professional Applications#

  • Event Documentation: Corporate events and team building activities
  • Project Milestones: Capture important project moments
  • Training Records: Document learning experiences and workshops
  • Client Interactions: Preserve significant business relationships

Therapeutic and Wellness#

  • Memory Therapy: Support for individuals with memory challenges
  • Mindfulness Practice: Intentional memory capture and reflection
  • Gratitude Journaling: Focus on positive experiences and emotions
  • Life Review: Systematic documentation of life experiences

Troubleshooting#

Common Issues#

Image Loading Problems:

  • Verify image URL accessibility and format support
  • Check internet connection and firewall settings
  • Ensure image file size is within reasonable limits

Conversation Flow Issues:

  • Provide clear, specific answers to AI questions
  • If stuck, try rephrasing or providing additional context
  • Restart the session if conversation becomes inconsistent

Memory Quality Concerns:

  • Be specific and detailed in your responses
  • Correct any misunderstandings immediately
  • Use follow-up sessions to refine captured memories

Friendzone Lite showcases the potential of combining visual AI with conversational intelligence to create meaningful, structured memory systems that preserve and organize life's important moments.