diff --git a/.github/workflows/python-syntax-check.yml b/.github/workflows/python-syntax-check.yml new file mode 100644 index 00000000..ab656e60 --- /dev/null +++ b/.github/workflows/python-syntax-check.yml @@ -0,0 +1,50 @@ +name: Python Syntax Check + +on: + pull_request: + branches: + - main + paths: + - 'application/single_app/**.py' + - '.github/workflows/python-syntax-check.yml' + +jobs: + syntax-check: + runs-on: ubuntu-latest + + steps: + - name: Checkout code + uses: actions/checkout@v4 + + - name: Set up Python + uses: actions/setup-python@v5 + with: + python-version: '3.11' + + - name: Run Python compilation check + run: | + cd application/single_app + echo "🔍 Running Python compilation checks on all .py files..." + failed_files=() + + for file in *.py; do + echo "" + echo "=== Compiling $file ===" + if python -m py_compile "$file" 2>&1; then + echo "✓ $file - OK" + else + echo "✗ $file - FAILED" + failed_files+=("$file") + fi + done + + echo "" + echo "================================" + if [ ${#failed_files[@]} -eq 0 ]; then + echo "✅ All Python files compiled successfully!" + exit 0 + else + echo "❌ ${#failed_files[@]} file(s) failed compilation:" + printf ' - %s\n' "${failed_files[@]}" + exit 1 + fi diff --git a/README.md b/README.md index 7f19f5ff..3f5dcb76 100644 --- a/README.md +++ b/README.md @@ -14,10 +14,111 @@ The application utilizes **Azure Cosmos DB** for storing conversations, metadata ## Quick Deploy -Use azd up [MORE DETAILS TO COME] +[Detailed deployment Guide](./deployers/bicep/README.md) +### Pre-Configuration: + +The following procedure must be completed with a user that has permissions to create an application registration in the users Entra tenant. + +#### Create the application registration: + +```powershell +cd ./deployers +``` + +Define your application name and your environment: + +``` +appName = +``` + +``` +environment = +``` + +The following script will create an Entra Enterprise Application, with an App Registration named *\*-*\*-ar for the web service called *\*-*\*-app. + +> [!TIP] +> +> The web service name may be overriden with the `-AppServceName` parameter. + +> [!TIP] +> +> A different expiration date for the secret which defaults to 180 days with the `-SecretExpirationDays` parameter. + +```powershell +.\Initialize-EntraApplication.ps1 -AppName "" -Environment "" -AppRolesJsonPath "./azurecli/appRegistrationRoles.json" ``` -azd up + +> [!NOTE] +> +> Be sure to save this information as it will not be available after the window is closed.* + +```======================================== +App Registration Created Successfully! +Application Name: +Client ID: +Tenant ID: +Service Principal ID: +Client Secret: +Secret Expiration: +``` + +In addition, the script will note additional steps that must be taken for the app registration step to be completed. + +1. Grant Admin Consent for API Permissions: + + - Navigate to Azure Portal > Entra ID > App registrations + - Find app: *\* + - Go to API permissions + - Click 'Grant admin consent for [Tenant]' + +2. Assign Users/Groups to Enterprise Application: + - Navigate to Azure Portal > Entra ID > Enterprise applications + - Find app: *\* + - Go to Users and groups + - Add user/group assignments with appropriate app roles + +3. Store the Client Secret Securely: + - Save the client secret in Azure Key Vault or secure credential store + - The secret value is shown above and will not be displayed again + +#### Configure AZD Environment + +Using the bash terminal in Visual Studio Code + +```powershell +cd ./deployers +``` + +If you work with other Azure clouds, you may need to update your cloud like `azd config set cloud.name AzureUSGovernment` - more information here - [Use Azure Developer CLI in sovereign clouds | Microsoft Learn](https://learn.microsoft.com/en-us/azure/developer/azure-developer-cli/sovereign-clouds) + +```powershell +azd config set cloud.name AzureCloud +``` + +This will open a browser window that the user with Owner level permissions to the target subscription will need to authenticate with. + +```powershell +azd auth login +``` + +Use the same value for the \ that was used in the application registration. + +```powershell +azd env new +``` + +Select the new environment + +```powershell +azd env select +``` + +This step will begin the deployment process. + +```powershell +Use azd up ``` ## Architecture @@ -27,50 +128,27 @@ azd up ## Features - **Chat with AI**: Interact with an AI model based on Azure OpenAI’s GPT and Thinking models. - - **RAG with Hybrid Search**: Upload documents and perform hybrid searches (vector + keyword), retrieving relevant information from your files to augment AI responses. - - **Document Management**: Upload, store, and manage multiple versions of documents—personal ("Your Workspace") or group-level ("Group Workspaces"). - - **Group Management**: Create and join groups to share access to group-specific documents, enabling collaboration with Role-Based Access Control (RBAC). - - **Ephemeral (Single-Convo) Documents**: Upload temporary documents available only during the current chat session, without persistent storage in Azure AI Search. - - **Conversation Archiving (Optional)**: Retain copies of user conversations—even after deletion from the UI—in a dedicated Cosmos DB container for audit, compliance, or legal requirements. - - **Content Safety (Optional)**: Integrate Azure AI Content Safety to review every user message *before* it reaches AI models, search indexes, or image generation services. Enforce custom filters and compliance policies, with an optional `SafetyAdmin` role for viewing violations. - - **Feedback System (Optional)**: Allow users to rate AI responses (thumbs up/down) and provide contextual comments on negative feedback. Includes user and admin dashboards, governed by an optional `FeedbackAdmin` role. - - **Bing Web Search (Optional)**: Augment AI responses with live Bing search results, providing up-to-date information. Configurable via Admin Settings. - - **Image Generation (Optional)**: Enable on-demand image creation using Azure OpenAI's DALL-E models, controlled via Admin Settings. - - **Video Extraction (Optional)**: Utilize Azure Video Indexer to transcribe speech and perform Optical Character Recognition (OCR) on video frames. Segments are timestamp-chunked for precise retrieval and enhanced citations linking back to the video timecode. - - **Audio Extraction (Optional)**: Leverage Azure Speech Service to transcribe audio files into timestamped text chunks, making audio content searchable and enabling enhanced citations linked to audio timecodes. - - **Document Classification (Optional)**: Admins define custom classification types and associated colors. Users tag uploaded documents with these labels, which flow through to AI conversations, providing lineage and insight into data sensitivity or type. - - **Enhanced Citation (Optional)**: Store processed, chunked files in Azure Storage (organized into user- and document-scoped folders). Display interactive citations in the UI—showing page numbers or timestamps—that link directly to the source document preview. - - **Metadata Extraction (Optional)**: Apply an AI model (configurable GPT model via Admin Settings) to automatically generate keywords, two-sentence summaries, and infer author/date for uploaded documents. Allows manual override for richer search context. - - **File Processing Logs (Optional)**: Enable verbose logging for all ingestion pipelines (workspaces and ephemeral chat uploads) to aid in debugging, monitoring, and auditing file processing steps. - - **Redis Cache (Optional)**: Integrate Azure Cache for Redis to provide a distributed, high-performance session store. This enables true horizontal scaling and high availability by decoupling user sessions from individual app instances. - - **Authentication & RBAC**: Secure access via Azure Active Directory (Entra ID) using MSAL. Supports Managed Identities for Azure service authentication, group-based controls, and custom application roles (`Admin`, `User`, `CreateGroup`, `SafetyAdmin`, `FeedbackAdmin`). - - **Supported File Types**: - - Text: `txt`, `md`, `html`, `json` - - * Documents: `pdf`, `docx`, `pptx`, `xlsx`, `xlsm`, `xls`, `csv` - * Images: `jpg`, `jpeg`, `png`, `bmp`, `tiff`, `tif`, `heif` - * Video: `mp4`, `mov`, `avi`, `wmv`, `mkv`, `webm` - * Audio: `mp3`, `wav`, `ogg`, `aac`, `flac`, `m4a` - -## Demos - -ADD DEMOS HERE \ No newline at end of file + - **Text**: `txt`, `md`, `html`, `json`, `xml`, `yaml`, `yml`, `log` + - **Documents**: `pdf`, `doc`, `docm`, `docx`, `pptx`, `xlsx`, `xlsm`, `xls`, `csv` + - **Images**: `jpg`, `jpeg`, `png`, `bmp`, `tiff`, `tif`, `heif` + - **Video**: `mp4`, `mov`, `avi`, `wmv`, `mkv`, `flv`, `mxf`, `gxf`, `ts`, `ps`, `3gp`, `3gpp`, `mpg`, `asf`, `m4v`, `isma`, `ismv`, `dvr-ms` + - **Audio**: `wav`, `m4a` \ No newline at end of file diff --git a/application/single_app/agent_logging_chat_completion_backup.py b/application/single_app/agent_logging_chat_completion_backup.py deleted file mode 100644 index 96afacf8..00000000 --- a/application/single_app/agent_logging_chat_completion_backup.py +++ /dev/null @@ -1,481 +0,0 @@ - - -import json -from pydantic import Field -from semantic_kernel.agents import ChatCompletionAgent -from functions_appinsights import log_event -import datetime -import re - - -class LoggingChatCompletionAgent(ChatCompletionAgent): - display_name: str | None = Field(default=None) - default_agent: bool = Field(default=False) - tool_invocations: list = Field(default_factory=list) - - def __init__(self, *args, display_name=None, default_agent=False, **kwargs): - # Remove these from kwargs so the base class doesn't see them - kwargs.pop('display_name', None) - kwargs.pop('default_agent', None) - super().__init__(*args, **kwargs) - self.display_name = display_name - self.default_agent = default_agent - # tool_invocations is now properly declared as a Pydantic field - - def log_tool_execution(self, tool_name, arguments=None, result=None): - """Manual method to log tool executions. Can be called by plugins.""" - tool_citation = { - "tool_name": tool_name, - "function_arguments": str(arguments) if arguments else "", - "function_result": str(result)[:500] if result else "", - "timestamp": datetime.datetime.utcnow().isoformat() - } - self.tool_invocations.append(tool_citation) - log_event( - f"[Agent Citations] Tool execution logged: {tool_name}", - extra={ - "agent": self.name, - "tool_name": tool_name, - "result_length": len(str(result)) if result else 0 - } - ) - - def patch_plugin_methods(self): - """ - DISABLED: Plugin method patching to prevent duplication. - Plugin logging is now handled by the @plugin_function_logger decorator system. - Citations are extracted from the plugin invocation logger in route_backend_chats.py. - """ - print(f"[Agent Logging] Skipping plugin method patching - using plugin invocation logger instead") - pass - - def infer_sql_query_from_context(self, user_question, response_content): - """Infer the likely SQL query based on user question and response.""" - if not user_question or not response_content: - return None, None - - user_q = user_question.lower() - response = response_content.lower() - - # Pattern matching for common query types - if any(phrase in user_q for phrase in ['most played', 'most popular', 'played the most', 'highest number']): - if 'craps crazy' in response and '422' in response: - return ( - "SELECT GameName, COUNT(*) as PlayCount FROM CasinoGameInteractions GROUP BY GameName ORDER BY PlayCount DESC LIMIT 1", - "Query returned: GameName='Craps Crazy', PlayCount=422 (most played game in the database)" - ) - else: - return ( - "SELECT GameName, COUNT(*) as PlayCount FROM CasinoGameInteractions GROUP BY GameName ORDER BY PlayCount DESC", - f"Executed aggregation query to find most played games. Result: {response_content[:100]}" - ) - - elif any(phrase in user_q for phrase in ['least played', 'least popular', 'played the least']): - return ( - "SELECT GameName, COUNT(*) as PlayCount FROM CasinoGameInteractions GROUP BY GameName ORDER BY PlayCount ASC LIMIT 1", - f"Query to find least played game. Result: {response_content[:100]}" - ) - - elif any(phrase in user_q for phrase in ['total', 'count', 'how many']): - if 'game' in user_q: - return ( - "SELECT COUNT(DISTINCT GameName) as TotalGames FROM CasinoGameInteractions", - f"Count query executed. Result: {response_content[:100]}" - ) - else: - return ( - "SELECT COUNT(*) as TotalInteractions FROM CasinoGameInteractions", - f"Count query executed. Result: {response_content[:100]}" - ) - - elif any(phrase in user_q for phrase in ['average', 'mean']): - if any(word in user_q for word in ['bet', 'wager']): - return ( - "SELECT AVG(BetAmount) as AvgBet FROM CasinoGameInteractions WHERE BetAmount IS NOT NULL", - f"Average bet calculation. Result: {response_content[:100]}" - ) - elif any(word in user_q for word in ['win', 'winning']): - return ( - "SELECT AVG(WinAmount) as AvgWin FROM CasinoGameInteractions WHERE WinAmount IS NOT NULL", - f"Average win calculation. Result: {response_content[:100]}" - ) - - elif any(phrase in user_q for phrase in ['list', 'show', 'what are']): - if 'game' in user_q: - return ( - "SELECT DISTINCT GameName FROM CasinoGameInteractions ORDER BY GameName", - f"List of games query. Result: {response_content[:150]}" - ) - - # Default fallback - return ( - "SELECT * FROM CasinoGameInteractions WHERE 1=1 /* query inferred from context */", - f"Executed query based on user question: '{user_question}'. Result: {response_content[:100]}" - ) - - def extract_tool_invocations_from_history(self, chat_history): - """Extract tool invocations from chat history for citations.""" - tool_citations = [] - - if not chat_history: - return tool_citations - - try: - # Iterate through chat history to find function calls and responses - for message in chat_history: - # Check if message has function calls in various formats - if hasattr(message, 'items') and message.items: - for item in message.items: - # Look for function call content (standard SK format) - if hasattr(item, 'function_name') and hasattr(item, 'function_result'): - tool_citation = { - "tool_name": item.function_name, - "function_arguments": str(getattr(item, 'arguments', {})), - "function_result": str(item.function_result)[:500], # Limit result size - "timestamp": datetime.datetime.utcnow().isoformat() - } - tool_citations.append(tool_citation) - # Alternative: Check for function call in content - elif hasattr(item, 'function_call'): - func_call = item.function_call - tool_citation = { - "tool_name": getattr(func_call, 'name', 'unknown'), - "function_arguments": str(getattr(func_call, 'arguments', {})), - "function_result": "Function called", - "timestamp": datetime.datetime.utcnow().isoformat() - } - tool_citations.append(tool_citation) - # Check for function result content type - elif hasattr(item, 'content_type') and item.content_type == 'function_result': - tool_citation = { - "tool_name": getattr(item, 'name', 'unknown_function'), - "function_arguments": "", - "function_result": str(getattr(item, 'text', ''))[:500], - "timestamp": datetime.datetime.utcnow().isoformat() - } - tool_citations.append(tool_citation) - - # Check for function calls in message metadata or inner content - if hasattr(message, 'metadata') and message.metadata: - # Look for function call metadata - for key, value in message.metadata.items(): - if 'function' in key.lower() or 'tool' in key.lower(): - tool_citation = { - "tool_name": f"metadata_{key}", - "function_arguments": "", - "function_result": str(value)[:500], - "timestamp": datetime.datetime.utcnow().isoformat() - } - tool_citations.append(tool_citation) - - # Check message role for tool/function messages - if hasattr(message, 'role') and hasattr(message, 'name'): - if message.role.value in ['tool', 'function']: - tool_citation = { - "tool_name": message.name or 'unknown_tool', - "function_arguments": "", - "function_result": str(getattr(message, 'content', ''))[:500], - "timestamp": datetime.datetime.utcnow().isoformat() - } - tool_citations.append(tool_citation) - - # Check for tool content in message content - if hasattr(message, 'content') and isinstance(message.content, str): - # Look for tool execution patterns in content - if "function_name:" in message.content or "tool_name:" in message.content: - # Extract tool information from content - tool_citation = { - "tool_name": "extracted_from_content", - "function_arguments": "", - "function_result": message.content[:500], - "timestamp": datetime.datetime.utcnow().isoformat() - } - tool_citations.append(tool_citation) - - except Exception as e: - log_event( - "[Agent Citations] Error extracting tool invocations from chat history", - extra={"agent": self.name, "error": str(e)}, - level="WARNING" - ) - - return tool_citations - - async def invoke(self, *args, **kwargs): - # Clear previous tool invocations - self.tool_invocations = [] - - # Log the prompt/messages before sending to LLM - log_event( - "[Logging Agent Request] Agent LLM prompt", - extra={ - "agent": self.name, - "prompt": [m.content[:30] for m in args[0]] if args else None - } - ) - - print(f"[Logging Agent Request] Agent: {self.name}") - print(f"[Logging Agent Request] Prompt: {[m.content[:30] for m in args[0]] if args else None}") - - # Store user question context for better tool detection - if args and args[0] and hasattr(args[0][-1], 'content'): - self._user_question = args[0][-1].content - elif args and args[0] and isinstance(args[0][-1], dict) and 'content' in args[0][-1]: - self._user_question = args[0][-1]['content'] - - # Apply patching to capture function calls - try: - self.patch_plugin_methods() - except Exception as e: - log_event(f"[Agent Citations] Error applying plugin patches: {e}", level="WARNING") - - response = None - try: - # Store initial message count to detect new messages from tool usage - initial_message_count = len(args[0]) if args and args[0] else 0 - result = super().invoke(*args, **kwargs) - - print(f"[Logging Agent Request] Result: {result}") - - if hasattr(result, "__aiter__"): - # Streaming/async generator response - response_chunks = [] - async for chunk in result: - response_chunks.append(chunk) - response = response_chunks[-1] if response_chunks else None - else: - # Regular coroutine response - response = await result - - print(f"[Logging Agent Request] Response: {response}") - - # Store the response for analysis - self._last_response = response - # Try to capture tool invocations from multiple sources - self._capture_tool_invocations_comprehensive(args, response, initial_message_count) - # Fallback: If no tool_invocations were captured, log the main plugin output as a citation - if not self.tool_invocations and response and hasattr(response, 'content'): - self.tool_invocations.append({ - "tool_name": getattr(self, 'name', 'All Citations'), - "function_arguments": str(args[-1]) if args else "", - "function_result": str(response.content)[:500], - "timestamp": datetime.datetime.utcnow().isoformat() - }) - return response - finally: - usage = getattr(response, "usage", None) - log_event( - "[Logging Agent Response][Usage] Agent LLM response", - extra={ - "agent": self.name, - "response": str(response)[:100] if response else None, - "prompt_tokens": getattr(usage, "prompt_tokens", None), - "completion_tokens": getattr(usage, "completion_tokens", None), - "total_tokens": getattr(usage, "total_tokens", None), - "usage": str(usage) if usage else None, - "tool_invocations_count": len(self.tool_invocations) - } - ) - - def _capture_tool_invocations_comprehensive(self, args, response, initial_message_count): - """ - SIMPLIFIED: Tool invocation capture for agent citations. - Most citation data now comes from the plugin invocation logger system. - This method only provides basic fallback logging for edge cases. - """ - try: - # Only capture basic response information as fallback - if response and hasattr(response, 'content') and response.content: - # Create a simple fallback citation if no plugin data is available - tool_citation = { - "tool_name": getattr(self, 'name', 'Agent Response'), - "function_arguments": str(args[-1]) if args else "", - "function_result": str(response.content)[:500], - "timestamp": datetime.datetime.utcnow().isoformat() - } - # Only add if we don't already have tool invocations - if not self.tool_invocations: - self.tool_invocations.append(tool_citation) - - log_event( - "[Agent Citations] Simplified tool capture completed", - extra={ - "agent": self.name, - "fallback_citations": len(self.tool_invocations), - "note": "Primary citations come from plugin invocation logger" - } - ) - - except Exception as e: - log_event( - "[Agent Citations] Error in simplified tool capture", - extra={"agent": self.name, "error": str(e)}, - level="WARNING" - ) - - def _extract_from_new_messages(self, new_messages): - """DISABLED: Extract tool invocations from newly added messages.""" - pass # Plugin invocation logger handles this now - - def _extract_from_kernel_state(self): - """DISABLED: Extract tool invocations from kernel execution state.""" - pass # Plugin invocation logger handles this now - - def _extract_from_response_content(self, content): - """DISABLED: Extract tool invocations from response content analysis.""" - pass # Plugin invocation logger handles this now - - def detect_sql_plugin_usage_from_logs(self): - """DISABLED: Enhanced SQL plugin detection.""" - pass # Plugin invocation logger handles this now - "function_result": "Retrieved database schema including table CasinoGameInteractions with 14 columns: InteractionID (bigint, PK), PlayerID (int), GameID (int), GameName (nvarchar), InteractionType (nvarchar), BetAmount (decimal), WinAmount (decimal), InteractionTimestamp (datetime2), MachineID (nvarchar), SessionDurationSeconds (int), MarketingTag (nvarchar), StaffInteraction (bit), Location (nvarchar), InsertedAt (datetime2)", - "timestamp": datetime.datetime.utcnow().isoformat() - }) - sql_tools_detected.append({ - "tool_name": "sqlquerytest", - "function_arguments": "query: 'SELECT * FROM INFORMATION_SCHEMA.TABLES' and related schema queries", - "function_result": "Executed database schema retrieval queries to identify table structures, primary keys, and column definitions. Found 1 primary table: CasinoGameInteractions", - "timestamp": datetime.datetime.utcnow().isoformat() - }) - - # Method 3: Check kernel plugin state for SQL execution - if hasattr(self, 'kernel') and self.kernel and hasattr(self.kernel, 'plugins'): - for plugin_name, plugin in self.kernel.plugins.items(): - if 'sql' in plugin_name.lower(): - # Check for execution state in the plugin - for plugin_attr in dir(plugin): - # Filter out internal Python/Pydantic attributes - if any(skip_pattern in plugin_attr for skip_pattern in [ - '__', '_abc_', '_fields', '_config', 'pydantic', 'model_', - 'schema_', 'json_', 'dict_', 'parse_', 'copy_', 'construct' - ]): - continue - - if any(keyword in plugin_attr.lower() for keyword in ['result', 'execution', 'last', 'data', 'query', 'schema']): - try: - plugin_value = getattr(plugin, plugin_attr) - if plugin_value and not callable(plugin_value) and str(plugin_value) not in ['', 'None', None]: - # Only capture meaningful data - value_str = str(plugin_value) - if len(value_str) > 10 and not value_str.startswith('{'): # Skip small/empty objects - tool_name = "sqlschematest" if "schema" in plugin_attr.lower() else "sqlquerytest" - sql_tools_detected.append({ - "tool_name": tool_name, - "function_arguments": f"captured_from: {plugin_attr}", - "function_result": value_str[:400], - "timestamp": datetime.datetime.utcnow().isoformat() - }) - except Exception: - continue - - # Method 4: If we don't have specific data but know SQL agent was used, create enhanced placeholders - if hasattr(self, 'name') and 'sql' in self.name.lower() and not sql_tools_detected: - # Enhanced placeholders with more realistic data - sql_tools_detected.extend([ - { - "tool_name": "sqlschematest", - "function_arguments": "include_system_tables: False, table_filter: None", - "function_result": "Retrieved database schema including table CasinoGameInteractions with 14 columns: InteractionID (bigint, PK), PlayerID (int), GameID (int), GameName (nvarchar), InteractionType (nvarchar), BetAmount (decimal), WinAmount (decimal), InteractionTimestamp (datetime2), MachineID (nvarchar), SessionDurationSeconds (int), MarketingTag (nvarchar), StaffInteraction (bit), Location (nvarchar), InsertedAt (datetime2)", - "timestamp": datetime.datetime.utcnow().isoformat() - }, - { - "tool_name": "sqlquerytest", - "function_arguments": "query: 'SELECT * FROM INFORMATION_SCHEMA.TABLES' and related schema queries", - "function_result": "Executed database schema retrieval queries to identify table structures, primary keys, and column definitions. Found 1 primary table: CasinoGameInteractions", - "timestamp": datetime.datetime.utcnow().isoformat() - } - ]) - - self.tool_invocations.extend(sql_tools_detected) - - if sql_tools_detected: - log_event( - f"[Agent Citations] Enhanced SQL detection found {len(sql_tools_detected)} tool executions", - extra={ - "agent": self.name, - "detected_tools": [t['tool_name'] for t in sql_tools_detected], - "has_actual_data": any('CasinoGameInteractions' in t.get('function_result', '') for t in sql_tools_detected) - } - ) - - def _extract_from_agent_attributes(self): - """Extract tool invocations from agent attributes and state.""" - # Check for any attributes that might indicate plugin execution - for attr_name in dir(self): - if 'plugin' in attr_name.lower() or 'function' in attr_name.lower(): - try: - attr_value = getattr(self, attr_name) - if callable(attr_value): - continue # Skip methods - - # If it's a list or dict that might contain execution info - if isinstance(attr_value, (list, dict)) and attr_value: - tool_citation = { - "tool_name": f"agent_attribute_{attr_name}", - "function_arguments": "", - "function_result": str(attr_value)[:200], - "timestamp": datetime.datetime.utcnow().isoformat() - } - self.tool_invocations.append(tool_citation) - except Exception: - continue # Skip attributes that can't be accessed - - def _extract_from_kernel_logs(self): - """Extract tool invocations from kernel execution logs and function call history.""" - try: - # Check if the kernel has any plugin execution history or logs - if hasattr(self, 'kernel') and self.kernel: - # Check for plugin execution state - if hasattr(self.kernel, 'plugins') and self.kernel.plugins: - for plugin_name, plugin in self.kernel.plugins.items(): - if hasattr(plugin, '_last_execution') or hasattr(plugin, 'execution_log'): - tool_citation = { - "tool_name": plugin_name, - "function_arguments": "", - "function_result": f"Plugin {plugin_name} was executed", - "timestamp": datetime.datetime.utcnow().isoformat() - } - self.tool_invocations.append(tool_citation) - - # Check for function execution history on the kernel - if hasattr(self.kernel, 'function_invoking_handlers') or hasattr(self.kernel, 'function_invoked_handlers'): - # If we have function handlers, it means functions were likely called - # Try to capture any available execution state - for attr_name in dir(self.kernel): - if 'execute' in attr_name.lower() or 'invoke' in attr_name.lower(): - try: - attr_value = getattr(self.kernel, attr_name) - if not callable(attr_value) and str(attr_value) not in ['', 'None', None]: - tool_citation = { - "tool_name": f"kernel_{attr_name}", - "function_arguments": "", - "function_result": str(attr_value)[:200], - "timestamp": datetime.datetime.utcnow().isoformat() - } - self.tool_invocations.append(tool_citation) - except Exception: - continue - - # Check for any execution context in the current agent - for context_attr in ['_execution_context', '_function_results', '_plugin_results']: - if hasattr(self, context_attr): - try: - context_value = getattr(self, context_attr) - if context_value: - tool_citation = { - "tool_name": context_attr.replace('_', ''), - "function_arguments": "", - "function_result": str(context_value)[:300], - "timestamp": datetime.datetime.utcnow().isoformat() - } - self.tool_invocations.append(tool_citation) - except Exception: - continue - - except Exception as e: - log_event( - "[Agent Citations] Error extracting from kernel logs", - extra={"agent": self.name, "error": str(e)}, - level="WARNING" - ) - \ No newline at end of file diff --git a/application/single_app/agent_logging_chat_completion_clean.py b/application/single_app/agent_logging_chat_completion_clean.py deleted file mode 100644 index e1fd1834..00000000 --- a/application/single_app/agent_logging_chat_completion_clean.py +++ /dev/null @@ -1,217 +0,0 @@ - -import json -from pydantic import Field -from semantic_kernel.agents import ChatCompletionAgent -from functions_appinsights import log_event -import datetime -import re - - -class LoggingChatCompletionAgent(ChatCompletionAgent): - display_name: str | None = Field(default=None) - default_agent: bool = Field(default=False) - tool_invocations: list = Field(default_factory=list) - - def __init__(self, *args, display_name=None, default_agent=False, **kwargs): - # Remove these from kwargs so the base class doesn't see them - kwargs.pop('display_name', None) - kwargs.pop('default_agent', None) - super().__init__(*args, **kwargs) - self.display_name = display_name - self.default_agent = default_agent - # tool_invocations is now properly declared as a Pydantic field - - def log_tool_execution(self, tool_name, arguments=None, result=None): - """Manual method to log tool executions. Can be called by plugins.""" - tool_citation = { - "tool_name": tool_name, - "function_arguments": str(arguments) if arguments else "", - "function_result": str(result)[:500] if result else "", - "timestamp": datetime.datetime.utcnow().isoformat() - } - self.tool_invocations.append(tool_citation) - log_event( - f"[Agent Citations] Tool execution logged: {tool_name}", - extra={ - "agent": self.name, - "tool_name": tool_name, - "result_length": len(str(result)) if result else 0 - } - ) - - def patch_plugin_methods(self): - """ - DISABLED: Plugin method patching to prevent duplication. - Plugin logging is now handled by the @plugin_function_logger decorator system. - Citations are extracted from the plugin invocation logger in route_backend_chats.py. - """ - print(f"[Agent Logging] Skipping plugin method patching - using plugin invocation logger instead") - pass - - def infer_sql_query_from_context(self, user_question, response_content): - """Infer the likely SQL query based on user question and response.""" - if not user_question or not response_content: - return None, None - - user_q = user_question.lower() - response = response_content.lower() - - # Pattern matching for common query types - if any(phrase in user_q for phrase in ['most played', 'most popular', 'played the most', 'highest number']): - if 'craps crazy' in response and '422' in response: - return ( - "SELECT GameName, COUNT(*) as PlayCount FROM CasinoGameInteractions GROUP BY GameName ORDER BY PlayCount DESC LIMIT 1", - "Query returned: GameName='Craps Crazy', PlayCount=422 (most played game in the database)" - ) - else: - return ( - "SELECT GameName, COUNT(*) as PlayCount FROM CasinoGameInteractions GROUP BY GameName ORDER BY PlayCount DESC", - f"Executed aggregation query to find most played games. Result: {response_content[:100]}" - ) - - elif any(phrase in user_q for phrase in ['least played', 'least popular', 'played the least']): - return ( - "SELECT GameName, COUNT(*) as PlayCount FROM CasinoGameInteractions GROUP BY GameName ORDER BY PlayCount ASC LIMIT 1", - f"Query to find least played game. Result: {response_content[:100]}" - ) - - elif any(phrase in user_q for phrase in ['total', 'count', 'how many']): - if 'game' in user_q: - return ( - "SELECT COUNT(DISTINCT GameName) as TotalGames FROM CasinoGameInteractions", - f"Count query executed. Result: {response_content[:100]}" - ) - else: - return ( - "SELECT COUNT(*) as TotalInteractions FROM CasinoGameInteractions", - f"Count query executed. Result: {response_content[:100]}" - ) - - elif any(phrase in user_q for phrase in ['average', 'mean']): - if any(word in user_q for word in ['bet', 'wager']): - return ( - "SELECT AVG(BetAmount) as AvgBet FROM CasinoGameInteractions WHERE BetAmount IS NOT NULL", - f"Average bet calculation. Result: {response_content[:100]}" - ) - elif any(word in user_q for word in ['win', 'winning']): - return ( - "SELECT AVG(WinAmount) as AvgWin FROM CasinoGameInteractions WHERE WinAmount IS NOT NULL", - f"Average win calculation. Result: {response_content[:100]}" - ) - - elif any(phrase in user_q for phrase in ['list', 'show', 'what are']): - if 'game' in user_q: - return ( - "SELECT DISTINCT GameName FROM CasinoGameInteractions ORDER BY GameName", - f"List of games query. Result: {response_content[:150]}" - ) - - # Default fallback - return ( - "SELECT * FROM CasinoGameInteractions WHERE 1=1 /* query inferred from context */", - f"Executed query based on user question: '{user_question}'. Result: {response_content[:100]}" - ) - - def extract_tool_invocations_from_history(self, chat_history): - """ - SIMPLIFIED: Extract tool invocations from chat history for citations. - Most citation data now comes from the plugin invocation logger system. - """ - return [] # Plugin invocation logger handles this now - - async def invoke(self, *args, **kwargs): - # Clear previous tool invocations - self.tool_invocations = [] - - # Log the prompt/messages before sending to LLM - log_event( - "[Logging Agent Request] Agent LLM prompt", - extra={ - "agent": self.name, - "prompt": [m.content[:30] for m in args[0]] if args else None - } - ) - - print(f"[Logging Agent Request] Agent: {self.name}") - print(f"[Logging Agent Request] Prompt: {[m.content[:30] for m in args[0]] if args else None}") - - # Store user question context for better tool detection - if args and args[0] and hasattr(args[0][-1], 'content'): - self._user_question = args[0][-1].content - elif args and args[0] and isinstance(args[0][-1], dict) and 'content' in args[0][-1]: - self._user_question = args[0][-1]['content'] - - response = None - try: - # Store initial message count to detect new messages from tool usage - initial_message_count = len(args[0]) if args and args[0] else 0 - result = super().invoke(*args, **kwargs) - - print(f"[Logging Agent Request] Result: {result}") - - if hasattr(result, "__aiter__"): - # Streaming/async generator response - response_chunks = [] - async for chunk in result: - response_chunks.append(chunk) - response = response_chunks[-1] if response_chunks else None - else: - # Regular coroutine response - response = await result - - print(f"[Logging Agent Request] Response: {response}") - - # Store the response for analysis - self._last_response = response - # Simplified citation capture - primary citations come from plugin invocation logger - self._capture_tool_invocations_simplified(args, response) - - return response - finally: - usage = getattr(response, "usage", None) - log_event( - "[Logging Agent Response][Usage] Agent LLM response", - extra={ - "agent": self.name, - "response": str(response)[:100] if response else None, - "prompt_tokens": getattr(usage, "prompt_tokens", None), - "completion_tokens": getattr(usage, "completion_tokens", None), - "total_tokens": getattr(usage, "total_tokens", None), - "usage": str(usage) if usage else None, - "fallback_citations": len(self.tool_invocations) - } - ) - - def _capture_tool_invocations_simplified(self, args, response): - """ - SIMPLIFIED: Basic fallback citation capture. - Primary citations come from the plugin invocation logger system. - This only provides basic response logging for edge cases. - """ - try: - # Only create a basic fallback citation for the agent response - if response and hasattr(response, 'content') and response.content: - tool_citation = { - "tool_name": getattr(self, 'name', 'Agent Response'), - "function_arguments": str(args[-1].content) if args and hasattr(args[-1], 'content') else "", - "function_result": str(response.content)[:500], - "timestamp": datetime.datetime.utcnow().isoformat() - } - # Only add as a fallback - plugin logger citations take priority - self.tool_invocations.append(tool_citation) - - log_event( - "[Agent Citations] Simplified fallback citation created", - extra={ - "agent": self.name, - "fallback_citations": len(self.tool_invocations), - "note": "Primary citations from plugin invocation logger" - } - ) - - except Exception as e: - log_event( - "[Agent Citations] Error in simplified citation capture", - extra={"agent": self.name, "error": str(e)}, - level="WARNING" - ) diff --git a/application/single_app/config.py b/application/single_app/config.py index 6aaad78e..2573b720 100644 --- a/application/single_app/config.py +++ b/application/single_app/config.py @@ -88,7 +88,7 @@ EXECUTOR_TYPE = 'thread' EXECUTOR_MAX_WORKERS = 30 SESSION_TYPE = 'filesystem' -VERSION = "0.235.001" +VERSION = "0.235.002" SECRET_KEY = os.getenv('SECRET_KEY', 'dev-secret-key-change-in-production') diff --git a/application/single_app/functions_documents.py b/application/single_app/functions_documents.py index e48c0131..017b819f 100644 --- a/application/single_app/functions_documents.py +++ b/application/single_app/functions_documents.py @@ -3087,13 +3087,8 @@ def analyze_image_with_vision_model(image_path, user_id, document_id, settings): 'analysis': 'detailed analysis' } or None if vision analysis is disabled or fails """ -<<<<<<< HEAD debug_print(f"[VISION_ANALYSIS_V2] Function entry - document_id: {document_id}, user_id: {user_id}") -======= - if not settings.get('enable_multimodal_vision', False): - return None ->>>>>>> origin/main try: # Convert image to base64 @@ -3101,7 +3096,6 @@ def analyze_image_with_vision_model(image_path, user_id, document_id, settings): image_bytes = img_file.read() base64_image = base64.b64encode(image_bytes).decode('utf-8') -<<<<<<< HEAD image_size = len(image_bytes) base64_size = len(base64_image) debug_print(f"[VISION_ANALYSIS] Image conversion for {document_id}:") @@ -3116,13 +3110,6 @@ def analyze_image_with_vision_model(image_path, user_id, document_id, settings): # Get vision model settings vision_model = settings.get('multimodal_vision_model', 'gpt-4o') debug_print(f"[VISION_ANALYSIS] Vision model selected: {vision_model}") -======= - # Determine image mime type - mime_type = mimetypes.guess_type(image_path)[0] or 'image/jpeg' - - # Get vision model settings - vision_model = settings.get('multimodal_vision_model', 'gpt-4o') ->>>>>>> origin/main if not vision_model: print(f"Warning: Multi-modal vision enabled but no model selected") @@ -3130,7 +3117,6 @@ def analyze_image_with_vision_model(image_path, user_id, document_id, settings): # Initialize client (reuse GPT configuration) enable_gpt_apim = settings.get('enable_gpt_apim', False) -<<<<<<< HEAD debug_print(f"[VISION_ANALYSIS] Using APIM: {enable_gpt_apim}") if enable_gpt_apim: @@ -3143,19 +3129,11 @@ def analyze_image_with_vision_model(image_path, user_id, document_id, settings): gpt_client = AzureOpenAI( api_version=api_version, azure_endpoint=endpoint, -======= - - if enable_gpt_apim: - gpt_client = AzureOpenAI( - api_version=settings.get('azure_apim_gpt_api_version'), - azure_endpoint=settings.get('azure_apim_gpt_endpoint'), ->>>>>>> origin/main api_key=settings.get('azure_apim_gpt_subscription_key') ) else: # Use managed identity or key auth_type = settings.get('azure_openai_gpt_authentication_type', 'key') -<<<<<<< HEAD api_version = settings.get('azure_openai_gpt_api_version') endpoint = settings.get('azure_openai_gpt_endpoint') @@ -3164,39 +3142,26 @@ def analyze_image_with_vision_model(image_path, user_id, document_id, settings): debug_print(f" API Version: {api_version}") debug_print(f" Auth Type: {auth_type}") -======= ->>>>>>> origin/main if auth_type == 'managed_identity': token_provider = get_bearer_token_provider( DefaultAzureCredential(), cognitive_services_scope ) gpt_client = AzureOpenAI( -<<<<<<< HEAD api_version=api_version, azure_endpoint=endpoint, -======= - api_version=settings.get('azure_openai_gpt_api_version'), - azure_endpoint=settings.get('azure_openai_gpt_endpoint'), ->>>>>>> origin/main azure_ad_token_provider=token_provider ) else: gpt_client = AzureOpenAI( -<<<<<<< HEAD api_version=api_version, azure_endpoint=endpoint, -======= - api_version=settings.get('azure_openai_gpt_api_version'), - azure_endpoint=settings.get('azure_openai_gpt_endpoint'), ->>>>>>> origin/main api_key=settings.get('azure_openai_gpt_key') ) # Create vision prompt print(f"Analyzing image with vision model: {vision_model}") -<<<<<<< HEAD # Determine which token parameter to use based on model type # o-series and gpt-5 models require max_completion_tokens instead of max_tokens vision_model_lower = vision_model.lower() @@ -3222,17 +3187,6 @@ def analyze_image_with_vision_model(image_path, user_id, document_id, settings): Ensure your entire response is valid JSON. Include all four keys even if some are empty strings or empty arrays.""" else: prompt_text = """Analyze this image and provide: -======= - response = gpt_client.chat.completions.create( - model=vision_model, - messages=[ - { - "role": "user", - "content": [ - { - "type": "text", - "text": """Analyze this image and provide: ->>>>>>> origin/main 1. A detailed description of what you see 2. List any objects, people, or notable elements 3. Extract any visible text (OCR) @@ -3245,7 +3199,6 @@ def analyze_image_with_vision_model(image_path, user_id, document_id, settings): "text": "...", "analysis": "..." }""" -<<<<<<< HEAD api_params = { "model": vision_model, @@ -3256,8 +3209,6 @@ def analyze_image_with_vision_model(image_path, user_id, document_id, settings): { "type": "text", "text": prompt_text -======= ->>>>>>> origin/main }, { "type": "image_url", @@ -3267,7 +3218,6 @@ def analyze_image_with_vision_model(image_path, user_id, document_id, settings): } ] } -<<<<<<< HEAD ] } @@ -3305,16 +3255,10 @@ def analyze_image_with_vision_model(image_path, user_id, document_id, settings): # Check finish reason if hasattr(response.choices[0], 'finish_reason'): debug_print(f" Finish reason: {response.choices[0].finish_reason}") -======= - ], - max_tokens=1000 - ) ->>>>>>> origin/main # Parse response content = response.choices[0].message.content -<<<<<<< HEAD # Handle None content if content is None: print(f"[VISION_ANALYSIS_V2] ⚠️ Response content is None!") @@ -3344,14 +3288,10 @@ def analyze_image_with_vision_model(image_path, user_id, document_id, settings): has_code_fence = '```' in content debug_print(f" Starts with JSON bracket: {is_json_like}") debug_print(f" Contains code fence: {has_code_fence}") -======= - debug_print(f"[VISION_ANALYSIS] Raw response for {document_id}: {content[:500]}...") ->>>>>>> origin/main # Try to parse as JSON, fallback to raw text try: # Clean up potential markdown code fences -<<<<<<< HEAD debug_print(f"[VISION_ANALYSIS] Attempting to clean JSON code fences...") content_cleaned = clean_json_codeFence(content) debug_print(f" Cleaned length: {len(content_cleaned)} characters") @@ -3376,23 +3316,10 @@ def analyze_image_with_vision_model(image_path, user_id, document_id, settings): 'parse_failed': True } debug_print(f"[VISION_ANALYSIS] Created fallback structure with raw response") -======= - content_cleaned = clean_json_codeFence(content) - vision_analysis = json.loads(content_cleaned) - debug_print(f"[VISION_ANALYSIS] Parsed JSON successfully for {document_id}") - except Exception as parse_error: - debug_print(f"[VISION_ANALYSIS] Vision response not valid JSON: {parse_error}") - print(f"Vision response not valid JSON, using raw text") - vision_analysis = { - 'description': content, - 'raw_response': content - } ->>>>>>> origin/main # Add model info to analysis vision_analysis['model'] = vision_model -<<<<<<< HEAD debug_print(f"[VISION_ANALYSIS] Final analysis structure for {document_id}:") debug_print(f" Model: {vision_model}") debug_print(f" Has 'description': {'description' in vision_analysis}") @@ -3414,13 +3341,6 @@ def analyze_image_with_vision_model(image_path, user_id, document_id, settings): txt = vision_analysis['text'] debug_print(f" Text length: {len(txt) if txt else 0} chars") debug_print(f" Text preview: {txt[:100] if txt else 'None'}...") -======= - debug_print(f"[VISION_ANALYSIS] Complete analysis for {document_id}:") - debug_print(f" Model: {vision_model}") - debug_print(f" Description: {vision_analysis.get('description', 'N/A')[:200]}...") - debug_print(f" Objects: {vision_analysis.get('objects', [])}") - debug_print(f" Text: {vision_analysis.get('text', 'N/A')[:100]}...") ->>>>>>> origin/main print(f"Vision analysis completed for document: {document_id}") return vision_analysis @@ -5195,79 +5115,10 @@ def process_di_document(document_id, user_id, temp_file_path, original_filename, # Don't fail the whole proc, total_embedding_tokens, embedding_model_nameess, just update status update_callback(status=f"Processing complete (metadata extraction warning)") -<<<<<<< HEAD # Note: Vision analysis now happens BEFORE save_chunks (moved earlier in the flow) # This ensures vision_analysis is available in metadata when chunks are being saved return total_final_chunks_processed, total_embedding_tokens, embedding_model_name -======= - # --- Multi-Modal Vision Analysis (for images only) --- - if is_image and enable_enhanced_citations: - enable_multimodal_vision = settings.get('enable_multimodal_vision', False) - if enable_multimodal_vision: - try: - update_callback(status="Performing AI vision analysis...") - - vision_analysis = analyze_image_with_vision_model( - temp_file_path, - user_id, - document_id, - settings - ) - - if vision_analysis: - print(f"Vision analysis completed for image: {original_filename}") - - # Update document with vision analysis results - update_fields = { - 'vision_analysis': vision_analysis, - 'vision_description': vision_analysis.get('description', ''), - 'vision_objects': vision_analysis.get('objects', []), - 'vision_extracted_text': vision_analysis.get('text', ''), - 'status': "AI vision analysis completed" - } - update_callback(**update_fields) - - # Save vision analysis as separate blob for citations - vision_json_path = temp_file_path + '_vision.json' - try: - with open(vision_json_path, 'w', encoding='utf-8') as f: - json.dump(vision_analysis, f, indent=2) - - vision_blob_filename = f"{os.path.splitext(original_filename)[0]}_vision_analysis.json" - - upload_blob_args = { - "temp_file_path": vision_json_path, - "user_id": user_id, - "document_id": document_id, - "blob_filename": vision_blob_filename, - "update_callback": update_callback - } - - if is_public_workspace: - upload_blob_args["public_workspace_id"] = public_workspace_id - elif is_group: - upload_blob_args["group_id"] = group_id - - upload_to_blob(**upload_blob_args) - print(f"Vision analysis saved to blob storage: {vision_blob_filename}") - - finally: - if os.path.exists(vision_json_path): - os.remove(vision_json_path) - else: - print(f"Vision analysis returned no results for: {original_filename}") - update_callback(status="Vision analysis completed (no results)") - - except Exception as e: - print(f"Warning: Error in vision analysis for {document_id}: {str(e)}") - import traceback - traceback.print_exc() - # Don't fail the whole process, just update status - update_callback(status=f"Processing complete (vision analysis warning)") - - return total_final_chunks_processed ->>>>>>> origin/main def _get_content_type(path: str) -> str: ext = os.path.splitext(path)[1].lower() @@ -5572,7 +5423,6 @@ def update_doc_callback(**kwargs): args["group_id"] = group_id if file_ext == '.txt': -<<<<<<< HEAD result = process_txt(**{k: v for k, v in args.items() if k != "file_ext"}) # Handle tuple return (chunks, tokens, model_name) if isinstance(result, tuple) and len(result) == 3: @@ -5603,17 +5453,6 @@ def update_doc_callback(**kwargs): total_chunks_saved, total_embedding_tokens, embedding_model_name = result else: total_chunks_saved = result -======= - total_chunks_saved = process_txt(**{k: v for k, v in args.items() if k != "file_ext"}) - elif file_ext == '.xml': - total_chunks_saved = process_xml(**{k: v for k, v in args.items() if k != "file_ext"}) - elif file_ext in ('.yaml', '.yml'): - total_chunks_saved = process_yaml(**{k: v for k, v in args.items() if k != "file_ext"}) - elif file_ext == '.log': - total_chunks_saved = process_log(**{k: v for k, v in args.items() if k != "file_ext"}) - elif file_ext in ('.doc', '.docm'): - total_chunks_saved = process_doc(**{k: v for k, v in args.items() if k != "file_ext"}) ->>>>>>> origin/main elif file_ext == '.html': result = process_html(**{k: v for k, v in args.items() if k != "file_ext"}) if isinstance(result, tuple) and len(result) == 3: diff --git a/deployers/bicep/README.md b/deployers/bicep/README.md index 492d569e..c2c51bf0 100644 --- a/deployers/bicep/README.md +++ b/deployers/bicep/README.md @@ -89,6 +89,8 @@ Using the bash terminal in Visual Studio Code `cd ./deployers` +`azd config set cloud.name AzureCloud` - If you work with other Azure clouds, you may need to update your cloud like `azd config set cloud.name AzureUSGovernment` - more information here - [Use Azure Developer CLI in sovereign clouds | Microsoft Learn](https://learn.microsoft.com/en-us/azure/developer/azure-developer-cli/sovereign-clouds) + `azd auth login` - this will open a browser window that the user with Owner level permissions to the target subscription will need to authenticate with. `azd env new ` - Use the same value for the \ that was used in the application registration. @@ -177,7 +179,7 @@ User shoud now be able to fully use Simple Chat application. "selected": [], "all": [] }, - ``` + ``` with @@ -205,7 +207,7 @@ User shoud now be able to fully use Simple Chat application. "selected": [], "all": [] }, - ``` + ``` with @@ -220,7 +222,7 @@ User shoud now be able to fully use Simple Chat application. "modelName": "text-embedding-3-small" ] }, - ``` + ``` - Update settings in the Cosmos UI and click Save. - Refresh web page and you shound now be able to Test the GPT and Embedding models.