<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://jekyllrb.com/" version="3.10.0">Jekyll</generator><link href="https://metaproxy.co/feed.xml" rel="self" type="application/atom+xml" /><link href="https://metaproxy.co/" rel="alternate" type="text/html" /><updated>2026-05-03T10:13:23+00:00</updated><id>https://metaproxy.co/feed.xml</id><title type="html">Metaproxy technologies</title><subtitle>A website.</subtitle><author><name>Tomohiko Araki</name></author><entry><title type="html">Building an AI-Powered Omnichannel Contact Center with Next.js, Twilio, and Vertex AI</title><link href="https://metaproxy.co/ContactCenter/" rel="alternate" type="text/html" title="Building an AI-Powered Omnichannel Contact Center with Next.js, Twilio, and Vertex AI" /><published>2026-01-26T00:00:00+00:00</published><updated>2026-01-26T00:00:00+00:00</updated><id>https://metaproxy.co/ContactCenter</id><content type="html" xml:base="https://metaproxy.co/ContactCenter/"><![CDATA[<h1 id="metatel-ai-powered-omnichannel-contact-center">Metatel: AI-Powered Omnichannel Contact Center</h1>

<p><strong>Metatel</strong> is a production-ready omnichannel contact center system that integrates <strong>phone calls</strong>, <strong>SMS</strong>, and <strong>LINE messaging</strong> with AI-powered responses using <strong>Google Vertex AI (Gemini)</strong>. Built with <strong>Next.js</strong> and deployed on <strong>Google Cloud Run</strong>, it provides a unified customer communication platform with real-time admin dashboard.</p>

<blockquote>
  <p><strong>📞 Try it yourself!</strong> You can experience the AI-powered IVR system by calling our main line:</p>

  <p><a href="tel:+16176827774" style="display: inline-block; background: linear-gradient(135deg, #007bff, #0056b3); color: white; font-weight: bold; padding: 0.6em 1.2em; border-radius: 12px; text-decoration: none; box-shadow: 0 4px 15px rgba(0,123,255,0.4);">
📱 +1-617-682-7774
</a></p>
</blockquote>

<hr />

<h2 id="overview">Overview</h2>

<ol>
  <li><strong>Unified Customer Identity</strong>
    <ul>
      <li>Customers are identified across channels (phone, SMS, LINE) using a unified user ID system</li>
      <li>Phone numbers and LINE IDs are linked to create a single customer profile</li>
      <li>Conversation history is shared across all channels for contextual AI responses</li>
    </ul>
  </li>
  <li><strong>AI-Powered Responses with Vertex AI Gemini</strong>
    <ul>
      <li>Uses Google’s Gemini 3 Flash model for intelligent, context-aware responses</li>
      <li>Adapts communication style based on customer intent (quick answers vs. empathetic listening)</li>
      <li>Automatically extracts and registers phone numbers from conversations</li>
    </ul>
  </li>
  <li><strong>Multi-Channel Support</strong>
    <ul>
      <li><strong>Phone (Twilio)</strong>: IVR system with AI greeting, call transfer, and voicemail</li>
      <li><strong>SMS (Twilio)</strong>: Automated AI responses to incoming text messages</li>
      <li><strong>LINE</strong>: Full chatbot integration with typing indicators and read receipts</li>
      <li><strong>BYOC (Bring Your Own Carrier)</strong>: Support for SIP trunking with existing phone numbers</li>
    </ul>
  </li>
  <li><strong>Real-Time Admin Dashboard</strong>
    <ul>
      <li>Google OAuth authentication for secure access</li>
      <li>Live conversation view across all channels</li>
      <li>AI-generated customer profiles and analytics</li>
      <li>5-second auto-refresh for real-time updates</li>
    </ul>
  </li>
</ol>

<hr />

<h2 id="system-architecture">System Architecture</h2>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>┌─────────────────────────────────────────────────────────────────────────┐
│                           Customer Touchpoints                          │
├─────────────────┬─────────────────┬─────────────────┬──────────────────┤
│   Phone Call    │      SMS        │      LINE       │   BYOC/SIP       │
│   +1-617-xxx    │   +1-617-xxx    │  @company_bot   │  Your Number     │
└────────┬────────┴────────┬────────┴────────┬────────┴────────┬─────────┘
         │                 │                 │                 │
         ▼                 ▼                 ▼                 ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                        Twilio / LINE Platform                           │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐    │
│  │ Voice API   │  │ SMS API     │  │ Messaging   │  │ BYOC Trunk  │    │
│  │ TwiML       │  │ Webhook     │  │ API         │  │ SIP INVITE  │    │
│  └──────┬──────┘  └──────┬──────┘  └──────┬──────┘  └──────┬──────┘    │
└─────────┼────────────────┼────────────────┼────────────────┼───────────┘
          │                │                │                │
          ▼                ▼                ▼                ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                     Next.js Application (Cloud Run)                     │
│  ┌──────────────────────────────────────────────────────────────────┐  │
│  │                        API Routes (App Router)                    │  │
│  │  /api/voice/*      - Phone call handling (TwiML)                 │  │
│  │  /api/sms          - SMS webhook &amp; AI response                   │  │
│  │  /api/line-webhook - LINE message handling                       │  │
│  │  /api/byoc/voice/* - BYOC SIP trunk handling                     │  │
│  │  /api/admin/*      - Admin API (user list, details)              │  │
│  │  /api/auth/*       - Google OAuth flow                           │  │
│  └──────────────────────────────────────────────────────────────────┘  │
│  ┌──────────────────────────────────────────────────────────────────┐  │
│  │                         Shared Libraries                          │  │
│  │  lib/voice-handler.ts  - Unified voice call logic                │  │
│  │  lib/gemini.ts         - Vertex AI Gemini integration            │  │
│  │  lib/firestore.ts      - Database operations                     │  │
│  │  lib/twilio.ts         - Twilio client &amp; helpers                 │  │
│  │  lib/emergency-guard.ts - Safety guards (blacklist/whitelist)    │  │
│  └──────────────────────────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────────────────────────┘
          │                │                │
          ▼                ▼                ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                         Google Cloud Platform                           │
│  ┌─────────────────┐  ┌─────────────────┐  ┌─────────────────┐         │
│  │   Firestore     │  │   Vertex AI     │  │   Cloud Run     │         │
│  │   - users       │  │   Gemini 3      │  │   Container     │         │
│  │   - events      │  │   Flash         │  │   Deployment    │         │
│  │   - phone-links │  │                 │  │                 │         │
│  │   - line-links  │  │                 │  │                 │         │
│  └─────────────────┘  └─────────────────┘  └─────────────────┘         │
└─────────────────────────────────────────────────────────────────────────┘
</code></pre></div></div>

<hr />

<h2 id="database-design">Database Design</h2>

<h3 id="unified-user-model">Unified User Model</h3>

<p>The system uses a <strong>unified user identity</strong> approach where customers are identified by a single internal ID, regardless of how they contact you.</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>users/{auto-id}
├── lineUserId: string | null      # LINE user ID (if connected)
├── displayName: string | null     # Display name from LINE
├── phoneNumbers: string[]         # All associated phone numbers
├── createdAt: timestamp
└── updatedAt: timestamp

phone-links/{normalizedPhone}      # Reverse lookup: phone → user
└── userId: string

line-links/{lineUserId}            # Reverse lookup: LINE → user  
└── userId: string

events/{auto-id}                   # All interactions (unified history)
├── userId: string                 # Internal user ID
├── type: 'line_message' | 'phone_call' | 'sms'
├── timestamp: timestamp
├── phoneNumber: string | null
└── content: {
│     role?: 'user' | 'model'      # For messages
│     text?: string
│     direction?: 'inbound' | 'outbound'  # For calls
│   }
</code></pre></div></div>

<p>This design enables:</p>
<ul>
  <li><strong>Cross-channel history</strong>: When a customer calls after messaging on LINE, the AI knows the full context</li>
  <li><strong>Automatic linking</strong>: Phone numbers mentioned in LINE chat are automatically linked to the user</li>
  <li><strong>Efficient queries</strong>: Composite indexes on <code class="language-plaintext highlighter-rouge">(userId, timestamp)</code> for fast history retrieval</li>
</ul>

<hr />

<h2 id="ai-response-system">AI Response System</h2>

<h3 id="context-aware-conversations">Context-Aware Conversations</h3>

<p>The Gemini AI adapts its response style based on customer intent:</p>

<div class="language-typescript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// lib/gemini.ts - System prompt excerpt</span>

<span class="kd">const</span> <span class="nx">SYSTEM_PROMPT</span> <span class="o">=</span> <span class="s2">`You are the LINE customer support for Metaproxy technologies.

【Reading the Room】
Adapt your response based on customer intent:

■ Quick Resolution Type (just stating requests)
→ 1-2 sentences max. "Got it!" and done.

■ Needs Listening Type (troubled, seeking advice)  
→ Listen carefully. Show empathy. Ask one follow-up question.

【How to Identify】
Quick Type:
- "Please do X" - direct requests
- Just sends phone number
- Short, to-the-point messages

Listening Type:
- "I'm having trouble with..."
- Explains situation in detail
- "What should I do about...?"
`</span><span class="p">;</span>
</code></pre></div></div>

<h3 id="phone-number-extraction">Phone Number Extraction</h3>

<p>The AI automatically detects and registers phone numbers from conversations:</p>

<div class="language-typescript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">export</span> <span class="kd">function</span> <span class="nx">extractPhoneNumber</span><span class="p">(</span><span class="nx">text</span><span class="p">:</span> <span class="kr">string</span><span class="p">):</span> <span class="kr">string</span> <span class="o">|</span> <span class="kc">null</span> <span class="p">{</span>
  <span class="c1">// Japanese formats: 090-1234-5678, 09012345678, +81-90-1234-5678</span>
  <span class="kd">const</span> <span class="nx">patterns</span> <span class="o">=</span> <span class="p">[</span>
    <span class="sr">/</span><span class="se">(?:\+</span><span class="sr">81</span><span class="se">[</span><span class="sr">-</span><span class="se">\s]?)?[</span><span class="sr">0-9</span><span class="se">]{2,4}[</span><span class="sr">-</span><span class="se">\s]?[</span><span class="sr">0-9</span><span class="se">]{2,4}[</span><span class="sr">-</span><span class="se">\s]?[</span><span class="sr">0-9</span><span class="se">]{3,4}</span><span class="sr">/g</span><span class="p">,</span>
    <span class="sr">/0</span><span class="se">[</span><span class="sr">0-9</span><span class="se">]{9,10}</span><span class="sr">/g</span><span class="p">,</span>
  <span class="p">];</span>
  
  <span class="k">for</span> <span class="p">(</span><span class="kd">const</span> <span class="nx">pattern</span> <span class="k">of</span> <span class="nx">patterns</span><span class="p">)</span> <span class="p">{</span>
    <span class="kd">const</span> <span class="nx">match</span> <span class="o">=</span> <span class="nx">text</span><span class="p">.</span><span class="nx">match</span><span class="p">(</span><span class="nx">pattern</span><span class="p">);</span>
    <span class="k">if</span> <span class="p">(</span><span class="nx">match</span><span class="p">)</span> <span class="p">{</span>
      <span class="k">return</span> <span class="nx">match</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="nx">replace</span><span class="p">(</span><span class="sr">/</span><span class="se">[</span><span class="sr">-</span><span class="se">\s]</span><span class="sr">/g</span><span class="p">,</span> <span class="dl">''</span><span class="p">);</span>
    <span class="p">}</span>
  <span class="p">}</span>
  <span class="k">return</span> <span class="kc">null</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div></div>

<hr />

<h2 id="voice-call-flow">Voice Call Flow</h2>

<h3 id="inbound-call-handling">Inbound Call Handling</h3>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Customer calls → Twilio webhook → /api/voice/route.ts
                                        │
                    ┌───────────────────┴───────────────────┐
                    ▼                                       ▼
            Business Hours?                          After Hours
                    │                                       │
                    ▼                                       ▼
            AI Greeting                              Voicemail
            "Hello! Press 1 for                      Recording
             sales, 2 for support..."                     │
                    │                                       ▼
                    ▼                                   Slack
            /api/voice/gather                        Notification
            (DTMF input)
                    │
        ┌───────────┼───────────┐
        ▼           ▼           ▼
    Press 1     Press 2     Press 9
    Transfer    Transfer    Emergency
    to Sales    to Support  Transfer
        │           │           │
        ▼           ▼           ▼
    /api/voice/dial-status
    (Call result handling)
</code></pre></div></div>

<h3 id="twiml-response-generation">TwiML Response Generation</h3>

<div class="language-typescript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// lib/voice-handler.ts</span>

<span class="k">export</span> <span class="k">async</span> <span class="kd">function</span> <span class="nx">handleVoiceCall</span><span class="p">(</span>
  <span class="nx">params</span><span class="p">:</span> <span class="p">{</span> <span class="nl">from</span><span class="p">:</span> <span class="kr">string</span><span class="p">;</span> <span class="nl">to</span><span class="p">:</span> <span class="kr">string</span><span class="p">;</span> <span class="nl">sipCallId</span><span class="p">?:</span> <span class="kr">string</span><span class="p">;</span> <span class="nl">byocTrunkSid</span><span class="p">?:</span> <span class="kr">string</span> <span class="p">},</span>
  <span class="nx">options</span><span class="p">:</span> <span class="nx">VoiceHandlerOptions</span> <span class="o">=</span> <span class="p">{}</span>
<span class="p">):</span> <span class="nb">Promise</span><span class="o">&lt;</span><span class="kr">string</span><span class="o">&gt;</span> <span class="p">{</span>
  <span class="kd">const</span> <span class="nx">response</span> <span class="o">=</span> <span class="nx">createVoiceResponse</span><span class="p">();</span>
  
  <span class="c1">// Check business hours</span>
  <span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="nx">isBusinessHours</span><span class="p">())</span> <span class="p">{</span>
    <span class="nx">response</span><span class="p">.</span><span class="nx">say</span><span class="p">({</span> <span class="na">language</span><span class="p">:</span> <span class="dl">'</span><span class="s1">ja-JP</span><span class="dl">'</span><span class="p">,</span> <span class="na">voice</span><span class="p">:</span> <span class="dl">'</span><span class="s1">Polly.Mizuki</span><span class="dl">'</span> <span class="p">},</span>
      <span class="dl">'</span><span class="s1">Thank you for calling. We are currently closed...</span><span class="dl">'</span><span class="p">);</span>
    <span class="nx">response</span><span class="p">.</span><span class="nx">record</span><span class="p">({</span> <span class="cm">/* voicemail settings */</span> <span class="p">});</span>
    <span class="k">return</span> <span class="nx">response</span><span class="p">.</span><span class="nx">toString</span><span class="p">();</span>
  <span class="p">}</span>
  
  <span class="c1">// Business hours: AI greeting with IVR</span>
  <span class="nx">response</span><span class="p">.</span><span class="nx">say</span><span class="p">({</span> <span class="na">language</span><span class="p">:</span> <span class="dl">'</span><span class="s1">ja-JP</span><span class="dl">'</span><span class="p">,</span> <span class="na">voice</span><span class="p">:</span> <span class="dl">'</span><span class="s1">Polly.Mizuki</span><span class="dl">'</span> <span class="p">},</span>
    <span class="dl">'</span><span class="s1">Thank you for calling Metaproxy technologies...</span><span class="dl">'</span><span class="p">);</span>
  
  <span class="nx">response</span><span class="p">.</span><span class="nx">gather</span><span class="p">({</span>
    <span class="na">numDigits</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span>
    <span class="na">action</span><span class="p">:</span> <span class="s2">`</span><span class="p">${</span><span class="nx">publicUrl</span><span class="p">}${</span><span class="nx">options</span><span class="p">.</span><span class="nx">gatherUrl</span><span class="p">}</span><span class="s2">`</span><span class="p">,</span>
    <span class="na">timeout</span><span class="p">:</span> <span class="mi">10</span><span class="p">,</span>
  <span class="p">}).</span><span class="nx">say</span><span class="p">({</span> <span class="na">language</span><span class="p">:</span> <span class="dl">'</span><span class="s1">ja-JP</span><span class="dl">'</span><span class="p">,</span> <span class="na">voice</span><span class="p">:</span> <span class="dl">'</span><span class="s1">Polly.Mizuki</span><span class="dl">'</span> <span class="p">},</span>
    <span class="dl">'</span><span class="s1">Press 1 for new inquiries, 2 for existing projects...</span><span class="dl">'</span><span class="p">);</span>
  
  <span class="k">return</span> <span class="nx">response</span><span class="p">.</span><span class="nx">toString</span><span class="p">();</span>
<span class="p">}</span>
</code></pre></div></div>

<hr />

<h2 id="safety-guards">Safety Guards</h2>

<h3 id="emergency-number-protection">Emergency Number Protection</h3>

<p>The system implements <strong>dual-layer protection</strong> against accidental calls/SMS to emergency numbers:</p>

<div class="language-typescript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// lib/emergency-guard.ts</span>

<span class="c1">// Layer 1: Blacklist (absolute block)</span>
<span class="kd">const</span> <span class="nx">BLACKLIST</span> <span class="o">=</span> <span class="k">new</span> <span class="nb">Set</span><span class="p">([</span>
  <span class="dl">'</span><span class="s1">911</span><span class="dl">'</span><span class="p">,</span> <span class="dl">'</span><span class="s1">110</span><span class="dl">'</span><span class="p">,</span> <span class="dl">'</span><span class="s1">119</span><span class="dl">'</span><span class="p">,</span> <span class="dl">'</span><span class="s1">118</span><span class="dl">'</span><span class="p">,</span>  <span class="c1">// US &amp; Japan emergency</span>
  <span class="dl">'</span><span class="s1">112</span><span class="dl">'</span><span class="p">,</span> <span class="dl">'</span><span class="s1">999</span><span class="dl">'</span><span class="p">,</span> <span class="dl">'</span><span class="s1">000</span><span class="dl">'</span><span class="p">,</span>         <span class="c1">// International emergency</span>
<span class="p">]);</span>

<span class="c1">// Layer 2: Whitelist (only these patterns allowed)</span>
<span class="kd">const</span> <span class="nx">WHITELIST_PATTERNS</span> <span class="o">=</span> <span class="p">[</span>
  <span class="sr">/^</span><span class="se">\+</span><span class="sr">81</span><span class="se">[</span><span class="sr">0-9</span><span class="se">]{9,10}</span><span class="sr">$/</span><span class="p">,</span>  <span class="c1">// Japan numbers only</span>
<span class="p">];</span>

<span class="c1">// Layer 3: Short number block</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">normalized</span><span class="p">.</span><span class="nx">length</span> <span class="o">&lt;=</span> <span class="mi">6</span><span class="p">)</span> <span class="p">{</span>
  <span class="k">return</span> <span class="p">{</span> <span class="na">blocked</span><span class="p">:</span> <span class="kc">true</span><span class="p">,</span> <span class="na">reason</span><span class="p">:</span> <span class="dl">'</span><span class="s1">Too short</span><span class="dl">'</span> <span class="p">};</span>
<span class="p">}</span>

<span class="c1">// For SMS: Additional US number block (A2P 10DLC compliance)</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">normalized</span><span class="p">.</span><span class="nx">startsWith</span><span class="p">(</span><span class="dl">'</span><span class="s1">+1</span><span class="dl">'</span><span class="p">))</span> <span class="p">{</span>
  <span class="k">return</span> <span class="p">{</span> <span class="na">blocked</span><span class="p">:</span> <span class="kc">true</span><span class="p">,</span> <span class="na">reason</span><span class="p">:</span> <span class="dl">'</span><span class="s1">US numbers blocked - A2P 10DLC not registered</span><span class="dl">'</span> <span class="p">};</span>
<span class="p">}</span>
</code></pre></div></div>

<hr />

<h2 id="admin-dashboard">Admin Dashboard</h2>

<h3 id="real-time-conversation-view">Real-Time Conversation View</h3>

<p>The admin dashboard provides a Discord-like interface for monitoring all customer interactions:</p>

<div class="language-typescript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// app/admin/page.tsx</span>

<span class="c1">// Auto-refresh every 5 seconds</span>
<span class="nx">useEffect</span><span class="p">(()</span> <span class="o">=&gt;</span> <span class="p">{</span>
  <span class="kd">const</span> <span class="nx">refreshInterval</span> <span class="o">=</span> <span class="nx">setInterval</span><span class="p">(()</span> <span class="o">=&gt;</span> <span class="p">{</span>
    <span class="nx">refreshUsersQuietly</span><span class="p">();</span>
    <span class="k">if</span> <span class="p">(</span><span class="nx">selectedUser</span><span class="p">)</span> <span class="p">{</span>
      <span class="nx">refreshUserDetailQuietly</span><span class="p">(</span><span class="nx">selectedUser</span><span class="p">.</span><span class="nx">id</span><span class="p">);</span>
    <span class="p">}</span>
  <span class="p">},</span> <span class="mi">5000</span><span class="p">);</span>
  <span class="k">return</span> <span class="p">()</span> <span class="o">=&gt;</span> <span class="nx">clearInterval</span><span class="p">(</span><span class="nx">refreshInterval</span><span class="p">);</span>
<span class="p">},</span> <span class="p">[</span><span class="nx">user</span><span class="p">,</span> <span class="nx">selectedUser</span><span class="p">]);</span>
</code></pre></div></div>

<h3 id="features">Features</h3>

<ul>
  <li><strong>Unified Timeline</strong>: See phone calls, SMS, and LINE messages in chronological order</li>
  <li><strong>Channel Badges</strong>: Visual indicators (📞 Phone, 📱 SMS, 💬 LINE) for each interaction</li>
  <li><strong>AI Customer Profiles</strong>: Automatically generated summaries of customer situation, interests</li>
  <li><strong>Contact Statistics</strong>: Call count, message count, last interaction dates</li>
</ul>

<hr />

<h2 id="deployment">Deployment</h2>

<h3 id="docker-configuration">Docker Configuration</h3>

<div class="language-dockerfile highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">FROM</span><span class="w"> </span><span class="s">node:22-alpine</span><span class="w"> </span><span class="k">AS</span><span class="w"> </span><span class="s">builder</span>
<span class="k">WORKDIR</span><span class="s"> /app</span>
<span class="k">COPY</span><span class="s"> package*.json ./</span>
<span class="k">RUN </span>npm ci
<span class="k">COPY</span><span class="s"> . .</span>
<span class="k">RUN </span>npm run build

<span class="k">FROM</span><span class="w"> </span><span class="s">node:22-alpine</span><span class="w"> </span><span class="k">AS</span><span class="w"> </span><span class="s">runner</span>
<span class="k">WORKDIR</span><span class="s"> /app</span>
<span class="k">ENV</span><span class="s"> NODE_ENV=production</span>
<span class="k">COPY</span><span class="s"> --from=builder /app/.next/standalone ./</span>
<span class="k">COPY</span><span class="s"> --from=builder /app/.next/static ./.next/static</span>
<span class="k">COPY</span><span class="s"> --from=builder /app/public ./public</span>
<span class="k">EXPOSE</span><span class="s"> 3000</span>
<span class="k">CMD</span><span class="s"> ["node", "server.js"]</span>
</code></pre></div></div>

<h3 id="environment-variables">Environment Variables</h3>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># Twilio</span>
<span class="nv">TWILIO_ACCOUNT_SID</span><span class="o">=</span>ACxxxx
<span class="nv">TWILIO_AUTH_TOKEN</span><span class="o">=</span>xxxx
<span class="nv">TWILIO_PHONE_NUMBER</span><span class="o">=</span>+1617xxxxxxx

<span class="c"># LINE</span>
<span class="nv">LINE_CHANNEL_SECRET</span><span class="o">=</span>xxxx
<span class="nv">LINE_CHANNEL_ACCESS_TOKEN</span><span class="o">=</span>xxxx

<span class="c"># Google Cloud</span>
<span class="nv">GCP_PROJECT_ID</span><span class="o">=</span>your-project
<span class="nv">GEMINI_MODEL</span><span class="o">=</span>gemini-3-flash-preview

<span class="c"># Application</span>
<span class="nv">PUBLIC_URL</span><span class="o">=</span>https://your-app.run.app
<span class="nv">FORWARD_PHONE_NUMBER</span><span class="o">=</span>+81xxxxxxxxxx
<span class="nv">ADMIN_EMAILS</span><span class="o">=</span>admin@example.com
</code></pre></div></div>

<h3 id="google-cloud-run-deployment">Google Cloud Run Deployment</h3>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># Build and push</span>
gcloud builds submit <span class="nt">--tag</span> gcr.io/PROJECT_ID/metatel

<span class="c"># Deploy</span>
gcloud run deploy metatel <span class="se">\</span>
  <span class="nt">--image</span> gcr.io/PROJECT_ID/metatel <span class="se">\</span>
  <span class="nt">--platform</span> managed <span class="se">\</span>
  <span class="nt">--region</span> asia-northeast1 <span class="se">\</span>
  <span class="nt">--allow-unauthenticated</span>
</code></pre></div></div>

<hr />

<h2 id="webhook-configuration">Webhook Configuration</h2>

<h3 id="twilio-console">Twilio Console</h3>

<table>
  <thead>
    <tr>
      <th>Channel</th>
      <th>Webhook URL</th>
      <th>Method</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>Voice (A call comes in)</td>
      <td><code class="language-plaintext highlighter-rouge">https://your-app.run.app/api/voice</code></td>
      <td>HTTP POST</td>
    </tr>
    <tr>
      <td>SMS (A message comes in)</td>
      <td><code class="language-plaintext highlighter-rouge">https://your-app.run.app/api/sms</code></td>
      <td>HTTP POST</td>
    </tr>
    <tr>
      <td>BYOC Trunk</td>
      <td><code class="language-plaintext highlighter-rouge">https://your-app.run.app/api/byoc/voice</code></td>
      <td>HTTP POST</td>
    </tr>
  </tbody>
</table>

<h3 id="line-developers-console">LINE Developers Console</h3>

<table>
  <thead>
    <tr>
      <th>Setting</th>
      <th>Value</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>Webhook URL</td>
      <td><code class="language-plaintext highlighter-rouge">https://your-app.run.app/api/line-webhook</code></td>
    </tr>
    <tr>
      <td>Use webhook</td>
      <td>Enabled</td>
    </tr>
    <tr>
      <td>Auto-reply</td>
      <td>Disabled</td>
    </tr>
  </tbody>
</table>

<hr />

<h2 id="key-features-summary">Key Features Summary</h2>

<table>
  <thead>
    <tr>
      <th>Feature</th>
      <th>Technology</th>
      <th>Description</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>Voice IVR</td>
      <td>Twilio TwiML</td>
      <td>Multi-level menu with DTMF input</td>
    </tr>
    <tr>
      <td>SMS Bot</td>
      <td>Twilio + Gemini</td>
      <td>AI-powered text responses</td>
    </tr>
    <tr>
      <td>LINE Bot</td>
      <td>LINE Messaging API + Gemini</td>
      <td>Full chatbot with typing indicators</td>
    </tr>
    <tr>
      <td>BYOC</td>
      <td>Twilio SIP Trunking</td>
      <td>Use your existing phone numbers</td>
    </tr>
    <tr>
      <td>Unified Identity</td>
      <td>Firestore</td>
      <td>Cross-channel customer profiles</td>
    </tr>
    <tr>
      <td>AI Responses</td>
      <td>Vertex AI Gemini 3</td>
      <td>Context-aware, style-adaptive replies</td>
    </tr>
    <tr>
      <td>Admin Dashboard</td>
      <td>Next.js + React</td>
      <td>Real-time conversation monitoring</td>
    </tr>
    <tr>
      <td>Safety Guards</td>
      <td>Blacklist/Whitelist</td>
      <td>Emergency number protection</td>
    </tr>
  </tbody>
</table>

<hr />

<h2 id="conclusion">Conclusion</h2>

<p>Metatel demonstrates how to build a production-ready omnichannel contact center using modern cloud technologies:</p>

<ul>
  <li><strong>Next.js App Router</strong> for serverless API routes</li>
  <li><strong>Twilio</strong> for voice and SMS capabilities</li>
  <li><strong>LINE Messaging API</strong> for chat integration</li>
  <li><strong>Google Vertex AI Gemini</strong> for intelligent, contextual responses</li>
  <li><strong>Firestore</strong> for real-time data synchronization</li>
  <li><strong>Cloud Run</strong> for scalable, containerized deployment</li>
</ul>

<p>The unified user identity model ensures customers receive consistent, contextual service regardless of which channel they use to reach you.</p>

<hr />

<h2 id="references">References</h2>

<ul>
  <li><a href="https://www.twilio.com/docs/voice/twiml">Twilio Voice TwiML</a></li>
  <li><a href="https://developers.line.biz/en/docs/messaging-api/">LINE Messaging API</a></li>
  <li><a href="https://cloud.google.com/vertex-ai/docs">Google Vertex AI</a></li>
  <li><a href="https://nextjs.org/docs/app">Next.js App Router</a></li>
  <li><a href="https://cloud.google.com/run/docs">Google Cloud Run</a></li>
</ul>]]></content><author><name>Tomohiko Araki</name></author><summary type="html"><![CDATA[Metatel: AI-Powered Omnichannel Contact Center]]></summary></entry><entry><title type="html">諏訪湖BRT構想 ― 車に依存しない街づくりのモデルケース</title><link href="https://metaproxy.co/suwakoBRT/" rel="alternate" type="text/html" title="諏訪湖BRT構想 ― 車に依存しない街づくりのモデルケース" /><published>2025-09-21T00:00:00+00:00</published><updated>2025-09-21T00:00:00+00:00</updated><id>https://metaproxy.co/suwakoBRT</id><content type="html" xml:base="https://metaproxy.co/suwakoBRT/"><![CDATA[<h2 id="概要">概要</h2>
<p>長野県の諏訪湖周辺は、観光資源（温泉・花火大会・美術館・湖畔レジャー）と、精密機械工業を中心とした産業基盤を併せ持つ地域である。しかし現在は「車社会」に強く依存しており、高齢者や未成年、観光客にとって移動の自由度は十分とは言えない。</p>

<p>そこで、諏訪湖を一周する <strong>BRT（Bus Rapid Transit）路線</strong> を整備し、地域の暮らし・観光・産業を支える新しい都市モデルを提案する。</p>

<hr />

<h2 id="基本計画">基本計画</h2>
<ul>
  <li><strong>ルート長</strong>：諏訪湖一周 約16km</li>
  <li><strong>停留所</strong>：徒歩15分圏ごとに1か所（約14停留所）</li>
  <li><strong>運行間隔</strong>：15分ごと（終日）</li>
  <li><strong>運行時間</strong>：6:00〜22:00</li>
  <li><strong>方式</strong>：湖岸道路に <strong>バスベイ方式の停留所</strong> を整備<br />
 → バス停車中も後続車の交通を妨げないため、定時性が大幅に向上</li>
  <li><strong>信号制御</strong>：主要交差点に優先信号を導入</li>
  <li><strong>車両</strong>：電動バス（将来的に自動運転導入を想定）</li>
</ul>

<p>👉 この仕組みにより、<strong>「15分に1本必ず来る高信頼の公共交通」</strong> が実現する。</p>

<hr />

<h2 id="初期投資と運営費概算">初期投資と運営費（概算）</h2>
<ul>
  <li><strong>初期投資</strong>：約25億円<br />
（停留所整備・車両5台・車庫・充電インフラ・ITシステム）</li>
  <li><strong>年間運営費</strong>：5億円前後<br />
（人件費・電力・整備・インフラ維持費）</li>
  <li><strong>20年間の総費用</strong>：120〜150億円</li>
</ul>

<h3 id="収支シミュレーション">収支シミュレーション</h3>
<ul>
  <li>収入：運賃200円 × 1日5,000人利用の場合 → 年間3.6億円</li>
  <li>運営費：年間5億円</li>
  <li>年間赤字：約1.4億円</li>
</ul>

<p>👉 利用者が1日7,000人を超えれば、<strong>ほぼ収支均衡</strong>に達する。<br />
👉 高校生・高齢者・観光客を合わせた潜在需要はそれ以上と見込まれ、<strong>「赤字覚悟で始めても、実際には大幅な赤字にはなりにくい」</strong> という構造になる。</p>

<hr />

<h2 id="期待される効果">期待される効果</h2>
<ol>
  <li><strong>高齢者・未成年の移動支援</strong>
    <ul>
      <li>免許返納後の高齢者や、通学世代（17歳以下）が自由に移動できる。</li>
      <li>特に高校生の通学利用は安定した基盤需要となる。</li>
    </ul>
  </li>
  <li><strong>観光回遊性の向上</strong>
    <ul>
      <li>湖畔の温泉地・美術館・花火大会を結ぶ「観光ループ」として機能。</li>
      <li>観光収入が交通事業の安定にも寄与する。</li>
    </ul>
  </li>
  <li><strong>環境負荷の低減</strong>
    <ul>
      <li>自家用車依存を減らし、渋滞・排ガスを抑制。景観・水質保全にも寄与。</li>
    </ul>
  </li>
  <li><strong>都市構造の再編</strong>
    <ul>
      <li>居住区を湖岸・駅周辺に集約することで、歩行＋公共交通で暮らせる都市圏を形成。</li>
    </ul>
  </li>
  <li><strong>未来型モデル都市</strong>
    <ul>
      <li>自動運転バスや電動化との親和性が高く、「地方都市の脱クルマ社会」の全国的モデルになり得る。</li>
    </ul>
  </li>
</ol>

<hr />

<h2 id="課題と留意点">課題と留意点</h2>
<ul>
  <li><strong>住民意識の転換</strong>：車社会から公共交通への移行には時間がかかる。</li>
  <li><strong>自治体間連携</strong>：諏訪市・岡谷市・下諏訪町などが協力して運営主体を確立する必要。</li>
  <li><strong>土地利用調整</strong>：停留所半径500m圏に住宅・商業・公共サービスを集約する都市計画が不可欠。</li>
</ul>

<hr />

<h2 id="結論">結論</h2>
<ul>
  <li>諏訪湖一周BRTは、<strong>200億円の基金を背景に赤字覚悟で運営すれば十分に実現可能</strong>。</li>
  <li>しかし実際には、高校生や高齢者の日常利用＋観光需要を取り込めば、<strong>大幅な赤字にはなりにくい堅実なモデル</strong>になる。</li>
  <li>バスベイ方式による停留所整備と15分ごとの定時運行で、<strong>「車がなくても暮らせる湖畔都市」</strong> を実現できる。</li>
</ul>

<p>👉 諏訪湖モデルは、全国の地方都市にとって「脱クルマ社会」の先駆けとなる可能性がある。</p>]]></content><author><name>Tomohiko Araki</name></author><summary type="html"><![CDATA[概要 長野県の諏訪湖周辺は、観光資源（温泉・花火大会・美術館・湖畔レジャー）と、精密機械工業を中心とした産業基盤を併せ持つ地域である。しかし現在は「車社会」に強く依存しており、高齢者や未成年、観光客にとって移動の自由度は十分とは言えない。]]></summary></entry><entry><title type="html">X(Twitter) Bot with AI and Web Content Fetching</title><link href="https://metaproxy.co/XBot/" rel="alternate" type="text/html" title="X(Twitter) Bot with AI and Web Content Fetching" /><published>2025-01-23T00:00:00+00:00</published><updated>2025-01-23T00:00:00+00:00</updated><id>https://metaproxy.co/XBot</id><content type="html" xml:base="https://metaproxy.co/XBot/"><![CDATA[<h1 id="mochi-bot-twitter-repository-explanation">mochi-bot-twitter Repository Explanation</h1>

<p><a href="https://github.com/rtree/mochi-bot-twitter"><strong>rtree/mochi-bot-twitter</strong></a> is a sample Python script repository that integrates <strong>OpenAI (ChatGPT)</strong> and <strong>X(Twitter)</strong>, while also incorporating <strong>Bing Search</strong> results to fetch the latest information. It demonstrates how to automatically tweet summaries of up-to-date content.</p>

<hr />

<h2 id="overview">Overview</h2>

<ol>
  <li><strong>Conversation History Management with OpenAI (ChatGPT API)</strong>
    <ul>
      <li>Uses a conversation history (a deque) to store both user inputs and AI responses.</li>
      <li>Each new user input is appended to this history, which is then sent to OpenAI’s API to generate replies that consider past dialogue.</li>
    </ul>
  </li>
  <li><strong>Incorporating Bing Search for Up-to-Date Information</strong>
    <ul>
      <li>The script extracts relevant keywords from the user’s query and searches them on Bing.</li>
      <li>It fetches the top few pages or PDF files, processes their contents, and generates a summary using ChatGPT.</li>
    </ul>
  </li>
  <li><strong>Posting Summaries to Twitter (via tweepy)</strong>
    <ul>
      <li>After generating a summarized response, the script can optionally post it as a series of tweets.</li>
      <li>It supports splitting a long response into multiple tweets in a thread.</li>
    </ul>
  </li>
</ol>

<hr />

<h2 id="high-level-flow">High-Level Flow</h2>

<ol>
  <li><strong>Load Environment Variables (<code class="language-plaintext highlighter-rouge">dotenv</code>)</strong>
    <ul>
      <li>Reads <code class="language-plaintext highlighter-rouge">.env</code> file to get <code class="language-plaintext highlighter-rouge">OPENAI_API_KEY</code>, <code class="language-plaintext highlighter-rouge">TWITTER_API_KEY</code>, and so on.</li>
      <li>Ensures each service has the required credentials for operation.</li>
    </ul>
  </li>
  <li><strong>Conversation History (<code class="language-plaintext highlighter-rouge">conversation_history</code>)</strong>
    <ul>
      <li>Maintained using a <code class="language-plaintext highlighter-rouge">deque</code> with a maximum length defined by <code class="language-plaintext highlighter-rouge">HISTORY_LENGTH</code>.</li>
      <li>Stores recent messages, ensuring context is preserved in the conversation.</li>
    </ul>
  </li>
  <li><strong>OpenAI API Integration (<code class="language-plaintext highlighter-rouge">openai</code> module)</strong>
    <ul>
      <li>A ChatGPT client is created via <code class="language-plaintext highlighter-rouge">OpenAI(api_key=OPENAI_API_KEY)</code>.</li>
      <li>For each user message, the script calls <code class="language-plaintext highlighter-rouge">client.chat.completions.create(...)</code> with the appropriate model (e.g., <code class="language-plaintext highlighter-rouge">gpt-3.5-turbo</code> or a specified model).</li>
    </ul>
  </li>
  <li><strong>Bing Search</strong>
    <ul>
      <li>A function <code class="language-plaintext highlighter-rouge">search_bing()</code> calls the Bing Search API (<code class="language-plaintext highlighter-rouge">https://api.bing.microsoft.com/v7.0/search</code>) using the keywords extracted from the user’s query.</li>
      <li>Up to <code class="language-plaintext highlighter-rouge">SEARCH_RESULTS</code> (default 8) top results are processed; their snippets or page contents are retrieved.</li>
    </ul>
  </li>
  <li><strong>Summarizing Results</strong>
    <ul>
      <li><code class="language-plaintext highlighter-rouge">summarize_results_with_pages_async()</code> fetches each page’s content (HTML, PDF, etc.), partially processes or extracts text, and concatenates the results.</li>
      <li>The concatenated text is then summarized by ChatGPT, producing a final answer that blends both the search results and the AI’s own reasoning.</li>
    </ul>
  </li>
  <li><strong>Twitter Posting (Optional)</strong>
    <ul>
      <li>If <code class="language-plaintext highlighter-rouge">TWITTER_DO_TWEET = True</code>, the resulting summary is split into multiple tweets, each posted as a reply to the previous one (a tweet thread).</li>
      <li>The code uses the <code class="language-plaintext highlighter-rouge">tweepy</code> library for tweeting.</li>
    </ul>
  </li>
  <li><strong>Main Function Execution</strong>
    <ul>
      <li>When <code class="language-plaintext highlighter-rouge">python main.py</code> is run, it triggers <code class="language-plaintext highlighter-rouge">asyncio.run(run_bot())</code>.</li>
      <li>A sample user query is defined (<code class="language-plaintext highlighter-rouge">"Summarize today’s news ..."</code>), Bing Search is performed if needed, the AI provides a response, and (optionally) tweets are posted.</li>
    </ul>
  </li>
</ol>

<hr />

<h2 id="code-structure">Code Structure</h2>

<p>Below is the core Python script from the repository for reference:</p>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">os</span>
<span class="kn">import</span> <span class="nn">tweepy</span>
<span class="kn">from</span> <span class="nn">dotenv</span> <span class="kn">import</span> <span class="n">load_dotenv</span>
<span class="kn">from</span> <span class="nn">openai</span> <span class="kn">import</span> <span class="n">OpenAI</span>
<span class="kn">import</span> <span class="nn">requests</span>
<span class="kn">from</span> <span class="nn">bs4</span> <span class="kn">import</span> <span class="n">BeautifulSoup</span>
<span class="kn">from</span> <span class="nn">collections</span> <span class="kn">import</span> <span class="n">deque</span>
<span class="kn">import</span> <span class="nn">asyncio</span>
<span class="kn">from</span> <span class="nn">datetime</span> <span class="kn">import</span> <span class="n">datetime</span>
<span class="kn">import</span> <span class="nn">lxml</span>
<span class="kn">from</span> <span class="nn">PyPDF2</span> <span class="kn">import</span> <span class="n">PdfReader</span>
<span class="kn">from</span> <span class="nn">io</span> <span class="kn">import</span> <span class="n">BytesIO</span>
<span class="kn">import</span> <span class="nn">base64</span>
<span class="kn">import</span> <span class="nn">sys</span>
<span class="kn">import</span> <span class="nn">logging</span>

<span class="c1"># Load environment variables
</span><span class="n">load_dotenv</span><span class="p">()</span>
<span class="n">OPENAI_API_KEY</span>       <span class="o">=</span> <span class="n">os</span><span class="p">.</span><span class="n">getenv</span><span class="p">(</span><span class="s">'OPENAI_API_KEY'</span><span class="p">)</span>
<span class="n">BING_API_KEY</span>         <span class="o">=</span> <span class="n">os</span><span class="p">.</span><span class="n">getenv</span><span class="p">(</span><span class="s">'BING_API_KEY'</span><span class="p">)</span>
<span class="n">HISTORY_LENGTH</span>       <span class="o">=</span> <span class="mi">10</span>
<span class="n">SEARCH_RESULTS</span>       <span class="o">=</span> <span class="mi">8</span>
<span class="n">SEARCH_MAX_CONTENT_LENGTH</span>   <span class="o">=</span> <span class="mi">5000</span>
<span class="n">TWITTER_API_KEY</span> <span class="o">=</span> <span class="n">os</span><span class="p">.</span><span class="n">getenv</span><span class="p">(</span><span class="s">'TWITTER_API_KEY'</span><span class="p">)</span>
<span class="n">TWITTER_API_SECRET</span> <span class="o">=</span> <span class="n">os</span><span class="p">.</span><span class="n">getenv</span><span class="p">(</span><span class="s">'TWITTER_API_SECRET'</span><span class="p">)</span>
<span class="n">TWITTER_ACCESS_TOKEN</span> <span class="o">=</span> <span class="n">os</span><span class="p">.</span><span class="n">getenv</span><span class="p">(</span><span class="s">'TWITTER_ACCESS_TOKEN'</span><span class="p">)</span>
<span class="n">TWITTER_ACCESS_SECRET</span> <span class="o">=</span> <span class="n">os</span><span class="p">.</span><span class="n">getenv</span><span class="p">(</span><span class="s">'TWITTER_ACCESS_SECRET'</span><span class="p">)</span>
<span class="n">TWITTER_BEARER_TOKEN</span>  <span class="o">=</span> <span class="n">os</span><span class="p">.</span><span class="n">getenv</span><span class="p">(</span><span class="s">'TWITTER_ACCESS_SECRET'</span><span class="p">)</span>
<span class="n">TWITTER_DO_TWEET</span>      <span class="o">=</span> <span class="bp">False</span>
<span class="n">TWITTER_DELIMITER</span>     <span class="o">=</span> <span class="s">"@@@@@@@@@@"</span>
<span class="n">REPUTABLE_DOMAINS</span> <span class="o">=</span> <span class="p">[</span>
    <span class="s">"go.jp"</span><span class="p">,</span> <span class="s">"gov"</span><span class="p">,</span>  <span class="c1"># Government and public sector
</span>    <span class="s">"scholar.google.com"</span><span class="p">,</span> <span class="s">"ci.nii.ac.jp"</span><span class="p">,</span> <span class="s">"pubmed.ncbi.nlm.nih.gov"</span><span class="p">,</span> <span class="s">"arxiv.org"</span><span class="p">,</span> <span class="s">"jstage.jst.go.jp"</span><span class="p">,</span> <span class="s">"ac.jp"</span><span class="p">,</span>  <span class="c1"># Academic and research databases
</span>    <span class="s">"nikkei.com"</span><span class="p">,</span>  <span class="c1"># News and business
</span>    <span class="s">"nature.com"</span><span class="p">,</span> <span class="s">"sciencedirect.com"</span><span class="p">,</span> <span class="s">"springer.com"</span><span class="p">,</span> <span class="s">"wiley.com"</span><span class="p">,</span>  <span class="c1"># Scientific publishers
</span>    <span class="s">"ieee.org"</span><span class="p">,</span> <span class="s">"researchgate.net"</span><span class="p">,</span>  <span class="c1"># Technical and engineering
</span>    <span class="s">"cambridge.org"</span><span class="p">,</span> <span class="s">"oxfordjournals.org"</span><span class="p">,</span>  <span class="c1"># Prestigious university publishers
</span>    <span class="s">"jamanetwork.com"</span><span class="p">,</span> <span class="s">"nejm.org"</span><span class="p">,</span> <span class="s">"plos.org"</span>  <span class="c1"># Medical and health research
</span><span class="p">]</span>

<span class="c1">#GPT_MODEL            = 'gpt-4-turbo-preview'
</span><span class="n">GPT_MODEL</span>            <span class="o">=</span> <span class="n">os</span><span class="p">.</span><span class="n">getenv</span><span class="p">(</span><span class="s">'GPT_MODEL'</span><span class="p">)</span>
<span class="n">AINAME</span>               <span class="o">=</span> <span class="s">"もちお"</span>
<span class="n">CHARACTER</span>            <span class="o">=</span> <span class="sa">f</span><span class="s">'You are an AI assistant cat named "</span><span class="si">{</span><span class="n">AINAME</span><span class="si">}</span><span class="s">", slightly mischievous and cute, speaking politely in Japanese with some playful endings like "だよ".'</span>

<span class="c1">###########################
#
# Initialize
</span>
<span class="c1"># Conversation history
</span><span class="n">conversation_history</span> <span class="o">=</span> <span class="n">deque</span><span class="p">(</span><span class="n">maxlen</span><span class="o">=</span><span class="n">HISTORY_LENGTH</span><span class="p">)</span>
<span class="c1"># Logging
</span><span class="n">log_dir</span> <span class="o">=</span> <span class="s">"./log"</span>
<span class="n">os</span><span class="p">.</span><span class="n">makedirs</span><span class="p">(</span><span class="n">log_dir</span><span class="p">,</span> <span class="n">exist_ok</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
<span class="n">log_file</span> <span class="o">=</span> <span class="n">os</span><span class="p">.</span><span class="n">path</span><span class="p">.</span><span class="n">join</span><span class="p">(</span><span class="n">log_dir</span><span class="p">,</span> <span class="sa">f</span><span class="s">"</span><span class="si">{</span><span class="n">datetime</span><span class="p">.</span><span class="n">today</span><span class="p">().</span><span class="n">strftime</span><span class="p">(</span><span class="s">"%Y-%m-%d"</span><span class="p">)</span><span class="si">}</span><span class="s">.log"</span><span class="p">)</span>
<span class="n">logging</span><span class="p">.</span><span class="n">basicConfig</span><span class="p">(</span><span class="n">level</span><span class="o">=</span><span class="n">logging</span><span class="p">.</span><span class="n">INFO</span><span class="p">,</span>
                    <span class="nb">format</span><span class="o">=</span><span class="s">"%(asctime)s - %(levelname)s - %(message)s"</span><span class="p">,</span>
                    <span class="n">handlers</span><span class="o">=</span><span class="p">[</span>
                        <span class="n">logging</span><span class="p">.</span><span class="n">FileHandler</span><span class="p">(</span><span class="n">log_file</span><span class="p">,</span> <span class="n">mode</span><span class="o">=</span><span class="s">"a"</span><span class="p">),</span>
                        <span class="n">logging</span><span class="p">.</span><span class="n">StreamHandler</span><span class="p">()</span>
                    <span class="p">])</span>

<span class="c1"># OpenAI client
</span><span class="n">client</span> <span class="o">=</span> <span class="n">OpenAI</span><span class="p">(</span><span class="n">api_key</span><span class="o">=</span><span class="n">OPENAI_API_KEY</span><span class="p">)</span>

<span class="c1"># Twitter client
</span><span class="n">twclient</span> <span class="o">=</span> <span class="n">tweepy</span><span class="p">.</span><span class="n">Client</span><span class="p">(</span>
    <span class="n">bearer_token</span><span class="o">=</span><span class="n">TWITTER_BEARER_TOKEN</span><span class="p">,</span>
    <span class="n">consumer_key</span><span class="o">=</span><span class="n">TWITTER_API_KEY</span><span class="p">,</span>
    <span class="n">consumer_secret</span><span class="o">=</span><span class="n">TWITTER_API_SECRET</span><span class="p">,</span>
    <span class="n">access_token</span><span class="o">=</span><span class="n">TWITTER_ACCESS_TOKEN</span><span class="p">,</span>
    <span class="n">access_token_secret</span><span class="o">=</span><span class="n">TWITTER_ACCESS_SECRET</span>
<span class="p">)</span>

<span class="c1"># -------------------------------- Search related ----------------------------
</span>
<span class="k">def</span> <span class="nf">parse_prompt</span><span class="p">(</span><span class="n">discIn</span><span class="p">):</span>
    <span class="n">p_src</span> <span class="o">=</span> <span class="p">(</span><span class="s">"You are an assistant that analyzes the user's prompt. "</span>
             <span class="s">"Please extract the main subject, subtopics, and relevant keywords from the recent user input."</span><span class="p">)</span>
    <span class="n">messages</span> <span class="o">=</span> <span class="p">[]</span>
    <span class="n">messages</span><span class="p">.</span><span class="n">extend</span><span class="p">(</span><span class="n">conversation_history</span><span class="p">)</span>
    <span class="n">messages</span><span class="p">.</span><span class="n">append</span><span class="p">({</span><span class="s">"role"</span><span class="p">:</span> <span class="s">"user"</span><span class="p">,</span> <span class="s">"content"</span><span class="p">:</span> <span class="n">p_src</span><span class="p">})</span>
    <span class="n">response</span> <span class="o">=</span> <span class="n">client</span><span class="p">.</span><span class="n">chat</span><span class="p">.</span><span class="n">completions</span><span class="p">.</span><span class="n">create</span><span class="p">(</span>
        <span class="n">model</span><span class="o">=</span><span class="n">GPT_MODEL</span><span class="p">,</span>
        <span class="n">messages</span><span class="o">=</span><span class="n">messages</span>
    <span class="p">)</span>
    <span class="n">logging</span><span class="p">.</span><span class="n">info</span><span class="p">(</span><span class="s">"= parse_prompt ============================================"</span><span class="p">)</span>
    <span class="n">logging</span><span class="p">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s">"response: </span><span class="si">{</span><span class="n">response</span><span class="p">.</span><span class="n">choices</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="n">message</span><span class="p">.</span><span class="n">content</span><span class="si">}</span><span class="s">"</span><span class="p">)</span>
    <span class="n">logging</span><span class="p">.</span><span class="n">info</span><span class="p">(</span><span class="s">"= End of parse_prompt ====================================="</span><span class="p">)</span>

    <span class="k">return</span> <span class="n">response</span><span class="p">.</span><span class="n">choices</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="n">message</span><span class="p">.</span><span class="n">content</span>

<span class="k">def</span> <span class="nf">should_search</span><span class="p">(</span><span class="n">discIn</span><span class="p">):</span>
    <span class="n">p_src</span> <span class="o">=</span> <span class="p">(</span><span class="s">"You are a smart assistant. Analyze the conversation history to determine whether the user’s query "</span>
             <span class="s">"requires external, up-to-date information. If yes, return 'Yes' only."</span><span class="p">)</span>
    <span class="n">messages</span> <span class="o">=</span> <span class="p">[]</span>
    <span class="n">messages</span><span class="p">.</span><span class="n">extend</span><span class="p">(</span><span class="n">conversation_history</span><span class="p">)</span>
    <span class="n">messages</span><span class="p">.</span><span class="n">append</span><span class="p">({</span><span class="s">"role"</span><span class="p">:</span> <span class="s">"user"</span><span class="p">,</span> <span class="s">"content"</span><span class="p">:</span> <span class="n">p_src</span><span class="p">})</span>
    <span class="n">response</span> <span class="o">=</span> <span class="n">client</span><span class="p">.</span><span class="n">chat</span><span class="p">.</span><span class="n">completions</span><span class="p">.</span><span class="n">create</span><span class="p">(</span>
        <span class="n">model</span><span class="o">=</span><span class="n">GPT_MODEL</span><span class="p">,</span>
        <span class="n">messages</span><span class="o">=</span><span class="n">messages</span>
    <span class="p">)</span>
    <span class="n">logging</span><span class="p">.</span><span class="n">info</span><span class="p">(</span><span class="s">"= should_search ============================================"</span><span class="p">)</span>
    <span class="n">logging</span><span class="p">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s">"response: </span><span class="si">{</span><span class="n">response</span><span class="p">.</span><span class="n">choices</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="n">message</span><span class="p">.</span><span class="n">content</span><span class="si">}</span><span class="s">"</span><span class="p">)</span>
    <span class="n">logging</span><span class="p">.</span><span class="n">info</span><span class="p">(</span><span class="s">"= End of should_search ====================================="</span><span class="p">)</span>
    <span class="k">return</span> <span class="n">response</span><span class="p">.</span><span class="n">choices</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="n">message</span><span class="p">.</span><span class="n">content</span>

<span class="k">def</span> <span class="nf">extract_keywords</span><span class="p">(</span><span class="n">parsed_text</span><span class="p">):</span>
    <span class="n">p_src</span> <span class="o">=</span> <span class="p">(</span><span class="sa">f</span><span class="s">"You are an assistant that extracts concise search keywords from the analyzed prompt. "</span>
             <span class="sa">f</span><span class="s">"Output them separated by single spaces: </span><span class="si">{</span><span class="n">parsed_text</span><span class="si">}</span><span class="s">"</span><span class="p">)</span>
    <span class="n">messages</span> <span class="o">=</span> <span class="p">[]</span>
    <span class="n">messages</span><span class="p">.</span><span class="n">extend</span><span class="p">(</span><span class="n">conversation_history</span><span class="p">)</span>
    <span class="n">messages</span><span class="p">.</span><span class="n">append</span><span class="p">({</span><span class="s">"role"</span><span class="p">:</span> <span class="s">"user"</span><span class="p">,</span> <span class="s">"content"</span><span class="p">:</span> <span class="n">p_src</span><span class="p">})</span>
    <span class="n">response</span> <span class="o">=</span> <span class="n">client</span><span class="p">.</span><span class="n">chat</span><span class="p">.</span><span class="n">completions</span><span class="p">.</span><span class="n">create</span><span class="p">(</span>
        <span class="n">model</span><span class="o">=</span><span class="n">GPT_MODEL</span><span class="p">,</span>
        <span class="n">messages</span><span class="o">=</span><span class="n">messages</span>
    <span class="p">)</span>
    <span class="n">logging</span><span class="p">.</span><span class="n">info</span><span class="p">(</span><span class="s">"= extract_keywords ============================================"</span><span class="p">)</span>
    <span class="n">logging</span><span class="p">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s">"response: </span><span class="si">{</span><span class="n">response</span><span class="p">.</span><span class="n">choices</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="n">message</span><span class="p">.</span><span class="n">content</span><span class="si">}</span><span class="s">"</span><span class="p">)</span>
    <span class="n">logging</span><span class="p">.</span><span class="n">info</span><span class="p">(</span><span class="s">"= End of extract_keywords ====================================="</span><span class="p">)</span>

    <span class="k">return</span> <span class="n">response</span><span class="p">.</span><span class="n">choices</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="n">message</span><span class="p">.</span><span class="n">content</span>

<span class="k">def</span> <span class="nf">search_bing</span><span class="p">(</span><span class="n">query</span><span class="p">,</span> <span class="n">domains</span><span class="o">=</span><span class="n">REPUTABLE_DOMAINS</span><span class="p">,</span> <span class="n">count</span><span class="o">=</span><span class="n">SEARCH_RESULTS</span><span class="p">):</span>
    <span class="n">url</span> <span class="o">=</span> <span class="s">"https://api.bing.microsoft.com/v7.0/search"</span>
    <span class="n">headers</span> <span class="o">=</span> <span class="p">{</span><span class="s">"Ocp-Apim-Subscription-Key"</span><span class="p">:</span> <span class="n">BING_API_KEY</span><span class="p">}</span>
    <span class="n">params</span> <span class="o">=</span> <span class="p">{</span><span class="s">"q"</span><span class="p">:</span> <span class="n">query</span><span class="p">,</span> <span class="s">"count"</span><span class="p">:</span> <span class="n">count</span><span class="p">,</span> <span class="s">"mkt"</span><span class="p">:</span> <span class="s">"en-US"</span><span class="p">,</span> <span class="s">"freshness"</span><span class="p">:</span> <span class="s">"Day"</span><span class="p">,</span> <span class="s">"sortBy"</span><span class="p">:</span> <span class="s">"Date"</span><span class="p">}</span>
    <span class="n">response</span> <span class="o">=</span> <span class="n">requests</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="p">,</span> <span class="n">headers</span><span class="o">=</span><span class="n">headers</span><span class="p">,</span> <span class="n">params</span><span class="o">=</span><span class="n">params</span><span class="p">)</span>
    <span class="n">response</span><span class="p">.</span><span class="n">raise_for_status</span><span class="p">()</span>
    <span class="n">search_data</span> <span class="o">=</span> <span class="n">response</span><span class="p">.</span><span class="n">json</span><span class="p">()</span>
    <span class="n">search_data</span><span class="p">[</span><span class="s">"urls"</span><span class="p">]</span> <span class="o">=</span> <span class="p">[</span><span class="n">result</span><span class="p">[</span><span class="s">"url"</span><span class="p">]</span> <span class="k">for</span> <span class="n">result</span> <span class="ow">in</span> <span class="n">search_data</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="s">"webPages"</span><span class="p">,</span> <span class="p">{}).</span><span class="n">get</span><span class="p">(</span><span class="s">"value"</span><span class="p">,</span> <span class="p">[])[:</span><span class="n">SEARCH_RESULTS</span><span class="p">]]</span>

    <span class="n">logging</span><span class="p">.</span><span class="n">info</span><span class="p">(</span><span class="s">"Bing Search Results:"</span><span class="p">)</span>
    <span class="k">for</span> <span class="n">result</span> <span class="ow">in</span> <span class="n">search_data</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="s">"webPages"</span><span class="p">,</span> <span class="p">{}).</span><span class="n">get</span><span class="p">(</span><span class="s">"value"</span><span class="p">,</span> <span class="p">[])[:</span><span class="n">count</span><span class="p">]:</span>
        <span class="n">logging</span><span class="p">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s">"Title: </span><span class="si">{</span><span class="n">result</span><span class="p">[</span><span class="s">'name'</span><span class="p">]</span><span class="si">}</span><span class="s">"</span><span class="p">)</span>
        <span class="n">logging</span><span class="p">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s">"URL: </span><span class="si">{</span><span class="n">result</span><span class="p">[</span><span class="s">'url'</span><span class="p">]</span><span class="si">}</span><span class="s">"</span><span class="p">)</span>
        <span class="n">logging</span><span class="p">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s">"Snippet: </span><span class="si">{</span><span class="n">result</span><span class="p">[</span><span class="s">'snippet'</span><span class="p">]</span><span class="si">}</span><span class="s">"</span><span class="p">)</span>
        <span class="n">logging</span><span class="p">.</span><span class="n">info</span><span class="p">(</span><span class="s">"---"</span><span class="p">)</span>
    <span class="k">return</span> <span class="n">search_data</span>

<span class="k">def</span> <span class="nf">fetch_page_content</span><span class="p">(</span><span class="n">url</span><span class="p">):</span>
    <span class="k">try</span><span class="p">:</span>
        <span class="n">response</span> <span class="o">=</span> <span class="n">requests</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="p">,</span> <span class="n">timeout</span><span class="o">=</span><span class="mi">10</span><span class="p">)</span>
        <span class="n">response</span><span class="p">.</span><span class="n">raise_for_status</span><span class="p">()</span>
        <span class="n">content_type</span> <span class="o">=</span> <span class="n">response</span><span class="p">.</span><span class="n">headers</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="s">"Content-Type"</span><span class="p">,</span> <span class="s">""</span><span class="p">)</span>

        <span class="k">if</span> <span class="s">"application/pdf"</span> <span class="ow">in</span> <span class="n">content_type</span><span class="p">:</span>
            <span class="n">pdf_reader</span> <span class="o">=</span> <span class="n">PdfReader</span><span class="p">(</span><span class="n">BytesIO</span><span class="p">(</span><span class="n">response</span><span class="p">.</span><span class="n">content</span><span class="p">))</span>
            <span class="n">pdf_text</span> <span class="o">=</span> <span class="s">""</span><span class="p">.</span><span class="n">join</span><span class="p">(</span><span class="n">page</span><span class="p">.</span><span class="n">extract_text</span><span class="p">()</span> <span class="k">for</span> <span class="n">page</span> <span class="ow">in</span> <span class="n">pdf_reader</span><span class="p">.</span><span class="n">pages</span><span class="p">)</span>
            <span class="k">return</span> <span class="n">pdf_text</span><span class="p">[:</span><span class="n">SEARCH_MAX_CONTENT_LENGTH</span><span class="p">],</span> <span class="s">"PDF"</span>
        <span class="k">elif</span> <span class="s">"text/html"</span> <span class="ow">in</span> <span class="n">content_type</span><span class="p">:</span>
            <span class="n">soup</span> <span class="o">=</span> <span class="n">BeautifulSoup</span><span class="p">(</span><span class="n">response</span><span class="p">.</span><span class="n">content</span><span class="p">,</span> <span class="s">"lxml"</span><span class="p">)</span>
            <span class="k">return</span> <span class="n">soup</span><span class="p">.</span><span class="n">get_text</span><span class="p">(</span><span class="n">separator</span><span class="o">=</span><span class="s">"</span><span class="se">\n</span><span class="s">"</span><span class="p">,</span> <span class="n">strip</span><span class="o">=</span><span class="bp">True</span><span class="p">)[:</span><span class="n">SEARCH_MAX_CONTENT_LENGTH</span><span class="p">],</span> <span class="s">"HTML"</span>
        <span class="k">elif</span> <span class="n">content_type</span><span class="p">.</span><span class="n">startswith</span><span class="p">(</span><span class="s">"image/"</span><span class="p">):</span>
            <span class="n">base64_img</span> <span class="o">=</span> <span class="n">base64</span><span class="p">.</span><span class="n">b64encode</span><span class="p">(</span><span class="n">response</span><span class="p">.</span><span class="n">content</span><span class="p">).</span><span class="n">decode</span><span class="p">(</span><span class="s">"utf-8"</span><span class="p">)</span>
            <span class="n">data_url</span> <span class="o">=</span> <span class="sa">f</span><span class="s">"data:</span><span class="si">{</span><span class="n">content_type</span><span class="si">}</span><span class="s">;base64,</span><span class="si">{</span><span class="n">base64_img</span><span class="si">}</span><span class="s">"</span>
            <span class="k">return</span> <span class="n">data_url</span><span class="p">,</span> <span class="s">"Image"</span>
        <span class="k">else</span><span class="p">:</span>
            <span class="k">return</span> <span class="bp">None</span><span class="p">,</span> <span class="s">"Unsupported"</span>
    <span class="k">except</span> <span class="nb">Exception</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span>
        <span class="n">logging</span><span class="p">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s">"Error fetching </span><span class="si">{</span><span class="n">url</span><span class="si">}</span><span class="s">: </span><span class="si">{</span><span class="nb">str</span><span class="p">(</span><span class="n">e</span><span class="p">)</span><span class="si">}</span><span class="s">"</span><span class="p">)</span>
        <span class="k">return</span> <span class="bp">None</span><span class="p">,</span> <span class="s">"Error"</span>

<span class="c1">################################################################################################################
</span>
<span class="k">async</span> <span class="k">def</span> <span class="nf">fetch_page_content_async</span><span class="p">(</span><span class="n">url</span><span class="p">):</span>
    <span class="s">"""
    Asynchronously fetch page content by offloading blocking I/O to a thread.
    Returns (content, type).
    """</span>
    <span class="k">def</span> <span class="nf">blocking_fetch</span><span class="p">():</span>
        <span class="k">try</span><span class="p">:</span>
            <span class="n">response</span> <span class="o">=</span> <span class="n">requests</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="p">,</span> <span class="n">timeout</span><span class="o">=</span><span class="mi">10</span><span class="p">)</span>
            <span class="n">response</span><span class="p">.</span><span class="n">raise_for_status</span><span class="p">()</span>
            <span class="n">content_type</span> <span class="o">=</span> <span class="n">response</span><span class="p">.</span><span class="n">headers</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="s">"Content-Type"</span><span class="p">,</span> <span class="s">""</span><span class="p">)</span>

            <span class="k">if</span> <span class="s">"application/pdf"</span> <span class="ow">in</span> <span class="n">content_type</span><span class="p">:</span>
                <span class="n">pdf_reader</span> <span class="o">=</span> <span class="n">PdfReader</span><span class="p">(</span><span class="n">BytesIO</span><span class="p">(</span><span class="n">response</span><span class="p">.</span><span class="n">content</span><span class="p">))</span>
                <span class="n">pdf_text</span> <span class="o">=</span> <span class="s">""</span><span class="p">.</span><span class="n">join</span><span class="p">(</span><span class="n">page</span><span class="p">.</span><span class="n">extract_text</span><span class="p">()</span> <span class="k">for</span> <span class="n">page</span> <span class="ow">in</span> <span class="n">pdf_reader</span><span class="p">.</span><span class="n">pages</span><span class="p">)</span>
                <span class="k">return</span> <span class="n">pdf_text</span><span class="p">[:</span><span class="n">SEARCH_MAX_CONTENT_LENGTH</span><span class="p">],</span> <span class="s">"PDF"</span>
            <span class="k">elif</span> <span class="s">"text/html"</span> <span class="ow">in</span> <span class="n">content_type</span><span class="p">:</span>
                <span class="n">soup</span> <span class="o">=</span> <span class="n">BeautifulSoup</span><span class="p">(</span><span class="n">response</span><span class="p">.</span><span class="n">content</span><span class="p">,</span> <span class="s">"lxml"</span><span class="p">)</span>
                <span class="n">text</span> <span class="o">=</span> <span class="n">soup</span><span class="p">.</span><span class="n">get_text</span><span class="p">(</span><span class="n">separator</span><span class="o">=</span><span class="s">"</span><span class="se">\n</span><span class="s">"</span><span class="p">,</span> <span class="n">strip</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
                <span class="k">return</span> <span class="n">text</span><span class="p">[:</span><span class="n">SEARCH_MAX_CONTENT_LENGTH</span><span class="p">],</span> <span class="s">"HTML"</span>
            <span class="k">elif</span> <span class="n">content_type</span><span class="p">.</span><span class="n">startswith</span><span class="p">(</span><span class="s">"image/"</span><span class="p">):</span>
                <span class="n">base64_img</span> <span class="o">=</span> <span class="n">base64</span><span class="p">.</span><span class="n">b64encode</span><span class="p">(</span><span class="n">response</span><span class="p">.</span><span class="n">content</span><span class="p">).</span><span class="n">decode</span><span class="p">(</span><span class="s">"utf-8"</span><span class="p">)</span>
                <span class="n">data_url</span> <span class="o">=</span> <span class="sa">f</span><span class="s">"data:</span><span class="si">{</span><span class="n">content_type</span><span class="si">}</span><span class="s">;base64,</span><span class="si">{</span><span class="n">base64_img</span><span class="si">}</span><span class="s">"</span>
                <span class="k">return</span> <span class="n">data_url</span><span class="p">,</span> <span class="s">"Image"</span>
            <span class="k">else</span><span class="p">:</span>
                <span class="k">return</span> <span class="bp">None</span><span class="p">,</span> <span class="s">"Unsupported"</span>
        <span class="k">except</span> <span class="nb">Exception</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span>
            <span class="n">logging</span><span class="p">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s">"Error fetching </span><span class="si">{</span><span class="n">url</span><span class="si">}</span><span class="s">: </span><span class="si">{</span><span class="nb">str</span><span class="p">(</span><span class="n">e</span><span class="p">)</span><span class="si">}</span><span class="s">"</span><span class="p">)</span>
            <span class="k">return</span> <span class="bp">None</span><span class="p">,</span> <span class="s">"Error"</span>

    <span class="n">content</span><span class="p">,</span> <span class="n">ctype</span> <span class="o">=</span> <span class="k">await</span> <span class="n">asyncio</span><span class="p">.</span><span class="n">to_thread</span><span class="p">(</span><span class="n">blocking_fetch</span><span class="p">)</span>
    <span class="k">return</span> <span class="n">content</span><span class="p">,</span> <span class="n">ctype</span>

<span class="c1">################################################################################################################
################################################################################################################
</span>
<span class="k">async</span> <span class="k">def</span> <span class="nf">summarize_results_with_pages_async</span><span class="p">(</span><span class="n">search_results</span><span class="p">):</span>
    <span class="s">"""
    Asynchronously fetch content for each search result. 
    Returns a combined string of all results (page or snippet).
    """</span>
    <span class="n">content_list</span> <span class="o">=</span> <span class="p">[]</span>
    <span class="n">web_results</span> <span class="o">=</span> <span class="n">search_results</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="s">"webPages"</span><span class="p">,</span> <span class="p">{}).</span><span class="n">get</span><span class="p">(</span><span class="s">"value"</span><span class="p">,</span> <span class="p">[])[:</span><span class="n">SEARCH_RESULTS</span><span class="p">]</span>

    <span class="n">tasks</span> <span class="o">=</span> <span class="p">[</span><span class="n">fetch_page_content_async</span><span class="p">(</span><span class="n">r</span><span class="p">[</span><span class="s">"url"</span><span class="p">])</span> <span class="k">for</span> <span class="n">r</span> <span class="ow">in</span> <span class="n">web_results</span><span class="p">]</span>
    <span class="n">pages</span> <span class="o">=</span> <span class="k">await</span> <span class="n">asyncio</span><span class="p">.</span><span class="n">gather</span><span class="p">(</span><span class="o">*</span><span class="n">tasks</span><span class="p">,</span> <span class="n">return_exceptions</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>

    <span class="k">for</span> <span class="p">(</span><span class="n">r</span><span class="p">,</span> <span class="n">page_result</span><span class="p">)</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">web_results</span><span class="p">,</span> <span class="n">pages</span><span class="p">):</span>
        <span class="n">title</span> <span class="o">=</span> <span class="n">r</span><span class="p">[</span><span class="s">"name"</span><span class="p">]</span>
        <span class="n">snippet</span> <span class="o">=</span> <span class="n">r</span><span class="p">[</span><span class="s">"snippet"</span><span class="p">]</span>
        <span class="n">url</span> <span class="o">=</span> <span class="n">r</span><span class="p">[</span><span class="s">"url"</span><span class="p">]</span>

        <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">page_result</span><span class="p">,</span> <span class="nb">Exception</span><span class="p">):</span>
            <span class="n">content_list</span><span class="p">.</span><span class="n">append</span><span class="p">(</span><span class="sa">f</span><span class="s">"Title: </span><span class="si">{</span><span class="n">title</span><span class="si">}</span><span class="se">\n</span><span class="s">URL: </span><span class="si">{</span><span class="n">url</span><span class="si">}</span><span class="se">\n</span><span class="s">Snippet:</span><span class="se">\n</span><span class="si">{</span><span class="n">snippet</span><span class="si">}</span><span class="se">\n</span><span class="s">"</span><span class="p">)</span>
            <span class="k">continue</span>

        <span class="n">page_content</span><span class="p">,</span> <span class="n">content_type</span> <span class="o">=</span> <span class="n">page_result</span>
        <span class="k">if</span> <span class="n">content_type</span> <span class="ow">in</span> <span class="p">(</span><span class="s">"HTML"</span><span class="p">,</span> <span class="s">"PDF"</span><span class="p">)</span> <span class="ow">and</span> <span class="n">page_content</span><span class="p">:</span>
            <span class="n">content_list</span><span class="p">.</span><span class="n">append</span><span class="p">(</span>
                <span class="sa">f</span><span class="s">"Title: </span><span class="si">{</span><span class="n">title</span><span class="si">}</span><span class="se">\n</span><span class="s">URL: </span><span class="si">{</span><span class="n">url</span><span class="si">}</span><span class="se">\n</span><span class="s">Content:</span><span class="se">\n</span><span class="si">{</span><span class="n">page_content</span><span class="si">}</span><span class="se">\n</span><span class="s">"</span>
            <span class="p">)</span>
        <span class="k">else</span><span class="p">:</span>
            <span class="n">content_list</span><span class="p">.</span><span class="n">append</span><span class="p">(</span>
                <span class="sa">f</span><span class="s">"Title: </span><span class="si">{</span><span class="n">title</span><span class="si">}</span><span class="se">\n</span><span class="s">URL: </span><span class="si">{</span><span class="n">url</span><span class="si">}</span><span class="se">\n</span><span class="s">Snippet:</span><span class="se">\n</span><span class="si">{</span><span class="n">snippet</span><span class="si">}</span><span class="se">\n</span><span class="s">"</span>
            <span class="p">)</span>

    <span class="k">return</span> <span class="s">"</span><span class="se">\n</span><span class="s">"</span><span class="p">.</span><span class="n">join</span><span class="p">(</span><span class="n">content_list</span><span class="p">)</span>

<span class="k">async</span> <span class="k">def</span> <span class="nf">summarize_results_async</span><span class="p">(</span><span class="n">search_results</span><span class="p">):</span>
    <span class="s">"""
    Calls GPT to summarize the combined content from search results.
    """</span>
    <span class="n">snippets</span> <span class="o">=</span> <span class="k">await</span> <span class="n">summarize_results_with_pages_async</span><span class="p">(</span><span class="n">search_results</span><span class="p">)</span>

    <span class="n">p_src</span> <span class="o">=</span> <span class="p">(</span>
        <span class="sa">f</span><span class="s">"</span><span class="si">{</span><span class="n">CHARACTER</span><span class="si">}</span><span class="s">. You are summarizing these search results as a report. "</span>
        <span class="s">"Please consider the conversation history, grasp the user's intent, "</span>
        <span class="s">"and summarize the following search results in English or Japanese, as needed. "</span>
        <span class="s">"Incorporate your own knowledge if it leads to a higher quality answer. "</span>
        <span class="sa">f</span><span class="s">"For Twitter, please split text around every 180 characters with the delimiter </span><span class="si">{</span><span class="n">TWITTER_DELIMITER</span><span class="si">}</span><span class="s">, etc. "</span>
        <span class="sa">f</span><span class="s">"Here is the data:</span><span class="se">\n</span><span class="si">{</span><span class="n">snippets</span><span class="si">}</span><span class="s">"</span>
    <span class="p">)</span>

    <span class="k">def</span> <span class="nf">blocking_chat_completion</span><span class="p">():</span>
        <span class="n">messages</span> <span class="o">=</span> <span class="p">[{</span><span class="s">"role"</span><span class="p">:</span> <span class="s">"system"</span><span class="p">,</span> <span class="s">"content"</span><span class="p">:</span> <span class="n">CHARACTER</span><span class="p">}]</span>
        <span class="n">messages</span><span class="p">.</span><span class="n">extend</span><span class="p">(</span><span class="n">conversation_history</span><span class="p">)</span>
        <span class="n">messages</span><span class="p">.</span><span class="n">append</span><span class="p">({</span><span class="s">"role"</span><span class="p">:</span> <span class="s">"user"</span><span class="p">,</span> <span class="s">"content"</span><span class="p">:</span> <span class="n">p_src</span><span class="p">})</span>

        <span class="k">return</span> <span class="n">client</span><span class="p">.</span><span class="n">chat</span><span class="p">.</span><span class="n">completions</span><span class="p">.</span><span class="n">create</span><span class="p">(</span>
            <span class="n">model</span><span class="o">=</span><span class="n">GPT_MODEL</span><span class="p">,</span>
            <span class="n">messages</span><span class="o">=</span><span class="n">messages</span>
        <span class="p">)</span>

    <span class="n">response</span> <span class="o">=</span> <span class="k">await</span> <span class="n">asyncio</span><span class="p">.</span><span class="n">to_thread</span><span class="p">(</span><span class="n">blocking_chat_completion</span><span class="p">)</span>
    <span class="n">summary</span> <span class="o">=</span> <span class="n">response</span><span class="p">.</span><span class="n">choices</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="n">message</span><span class="p">.</span><span class="n">content</span>

    <span class="n">titles</span> <span class="o">=</span> <span class="n">search_results</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="s">"titles"</span><span class="p">,</span> <span class="p">[])</span>
    <span class="n">urls</span> <span class="o">=</span> <span class="n">search_results</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="s">"urls"</span><span class="p">,</span> <span class="p">[])</span>
    <span class="n">sources</span> <span class="o">=</span> <span class="s">"</span><span class="se">\n</span><span class="s">"</span><span class="p">.</span><span class="n">join</span><span class="p">(</span><span class="sa">f</span><span class="s">"Source: </span><span class="si">{</span><span class="n">t</span><span class="si">}</span><span class="s"> - </span><span class="si">{</span><span class="n">u</span><span class="si">}</span><span class="s">"</span> <span class="k">for</span> <span class="n">t</span><span class="p">,</span> <span class="n">u</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">titles</span><span class="p">,</span> <span class="n">urls</span><span class="p">))</span>

    <span class="k">return</span> <span class="sa">f</span><span class="s">"</span><span class="si">{</span><span class="n">summary</span><span class="si">}</span><span class="se">\n\n</span><span class="si">{</span><span class="n">sources</span><span class="si">}</span><span class="s">"</span>

<span class="k">async</span> <span class="k">def</span> <span class="nf">search_or_call_openai_async</span><span class="p">(</span><span class="n">discIn</span><span class="p">,</span> <span class="n">img</span><span class="p">):</span>
    <span class="s">"""
    Decide whether external Bing search is needed; if yes, fetch pages concurrently,
    then summarize. Otherwise, call GPT directly.
    """</span>
    <span class="k">if</span> <span class="n">img</span> <span class="ow">or</span> <span class="nb">any</span><span class="p">(</span><span class="s">"http"</span> <span class="ow">in</span> <span class="n">entry</span><span class="p">[</span><span class="s">"content"</span><span class="p">]</span> <span class="k">for</span> <span class="n">entry</span> <span class="ow">in</span> <span class="n">discIn</span><span class="p">):</span>
        <span class="n">logging</span><span class="p">.</span><span class="n">info</span><span class="p">(</span><span class="s">"Skipping search and calling OpenAI directly."</span><span class="p">)</span>
        <span class="k">return</span> <span class="n">just_call_openai</span><span class="p">(</span><span class="n">discIn</span><span class="p">)</span>
    <span class="k">else</span><span class="p">:</span>
        <span class="n">yesorno</span> <span class="o">=</span> <span class="s">"Yes"</span>  <span class="c1"># Hardcoded example; normally would call should_search(discIn)
</span>        <span class="k">if</span> <span class="s">"Yes"</span> <span class="ow">in</span> <span class="n">yesorno</span><span class="p">:</span>
            <span class="n">logging</span><span class="p">.</span><span class="n">info</span><span class="p">(</span><span class="s">"searching... ---------------------------------------------"</span><span class="p">)</span>
            <span class="n">parsed_result</span> <span class="o">=</span> <span class="n">parse_prompt</span><span class="p">(</span><span class="n">discIn</span><span class="p">)</span>
            <span class="n">keywords</span> <span class="o">=</span> <span class="n">extract_keywords</span><span class="p">(</span><span class="n">parsed_result</span><span class="p">)</span>
            <span class="n">logging</span><span class="p">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s">"keyword: </span><span class="si">{</span><span class="n">keywords</span><span class="si">}</span><span class="s">"</span><span class="p">)</span>
            <span class="n">search_results</span> <span class="o">=</span> <span class="n">search_bing</span><span class="p">(</span><span class="n">keywords</span><span class="p">)</span>

            <span class="n">summary</span> <span class="o">=</span> <span class="k">await</span> <span class="n">summarize_results_async</span><span class="p">(</span><span class="n">search_results</span><span class="p">)</span>
            <span class="k">return</span> <span class="n">summary</span>
        <span class="k">else</span><span class="p">:</span>
            <span class="n">logging</span><span class="p">.</span><span class="n">info</span><span class="p">(</span><span class="s">"generating... --------------------------------------------"</span><span class="p">)</span>
            <span class="k">return</span> <span class="n">just_call_openai</span><span class="p">(</span><span class="n">discIn</span><span class="p">)</span>

<span class="k">async</span> <span class="k">def</span> <span class="nf">ai_respond</span><span class="p">(</span><span class="n">discIn</span><span class="p">,</span> <span class="n">img</span><span class="p">):</span>
    <span class="s">"""
    High-level function to produce an AI response (asynchronous).
    """</span>
    <span class="k">try</span><span class="p">:</span>
        <span class="n">result</span> <span class="o">=</span> <span class="k">await</span> <span class="n">search_or_call_openai_async</span><span class="p">(</span><span class="n">discIn</span><span class="p">,</span> <span class="n">img</span><span class="p">)</span>
        <span class="k">return</span> <span class="n">result</span>
    <span class="k">except</span> <span class="nb">Exception</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span>
        <span class="n">logging</span><span class="p">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s">"API Call Error: </span><span class="si">{</span><span class="nb">str</span><span class="p">(</span><span class="n">e</span><span class="p">)</span><span class="si">}</span><span class="s">"</span><span class="p">)</span>
        <span class="k">return</span> <span class="sa">f</span><span class="s">"Error: </span><span class="si">{</span><span class="nb">str</span><span class="p">(</span><span class="n">e</span><span class="p">)</span><span class="si">}</span><span class="s">"</span>

<span class="k">def</span> <span class="nf">just_call_openai</span><span class="p">(</span><span class="n">discIn</span><span class="p">):</span>
    <span class="n">messages</span> <span class="o">=</span> <span class="p">[{</span><span class="s">"role"</span><span class="p">:</span> <span class="s">"system"</span><span class="p">,</span> <span class="s">"content"</span><span class="p">:</span> <span class="sa">f</span><span class="s">"</span><span class="si">{</span><span class="n">CHARACTER</span><span class="si">}</span><span class="s">"</span><span class="p">}]</span>
    <span class="n">messages</span><span class="p">.</span><span class="n">extend</span><span class="p">(</span><span class="n">conversation_history</span><span class="p">)</span>
    <span class="n">completion</span> <span class="o">=</span> <span class="n">client</span><span class="p">.</span><span class="n">chat</span><span class="p">.</span><span class="n">completions</span><span class="p">.</span><span class="n">create</span><span class="p">(</span>
        <span class="n">model</span><span class="o">=</span><span class="n">GPT_MODEL</span><span class="p">,</span>
        <span class="n">messages</span><span class="o">=</span><span class="n">messages</span>
    <span class="p">)</span>
    <span class="k">return</span> <span class="n">completion</span><span class="p">.</span><span class="n">choices</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="n">message</span><span class="p">.</span><span class="n">content</span>

<span class="k">def</span> <span class="nf">post_to_twitter</span><span class="p">(</span><span class="n">content</span><span class="p">):</span>
    <span class="n">tweets</span> <span class="o">=</span> <span class="n">content</span><span class="p">.</span><span class="n">split</span><span class="p">(</span><span class="n">TWITTER_DELIMITER</span><span class="p">)</span>
    <span class="n">trimmed_tweets</span> <span class="o">=</span> <span class="p">[</span><span class="n">tweet</span><span class="p">[:</span><span class="mi">199</span><span class="p">]</span> <span class="k">for</span> <span class="n">tweet</span> <span class="ow">in</span> <span class="n">tweets</span><span class="p">]</span>

    <span class="n">first_tweet</span> <span class="o">=</span> <span class="n">twclient</span><span class="p">.</span><span class="n">create_tweet</span><span class="p">(</span><span class="n">text</span><span class="o">=</span><span class="n">trimmed_tweets</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span>
    <span class="n">tweet_id</span> <span class="o">=</span> <span class="n">first_tweet</span><span class="p">.</span><span class="n">data</span><span class="p">[</span><span class="s">"id"</span><span class="p">]</span>
    
    <span class="k">for</span> <span class="n">tweet</span> <span class="ow">in</span> <span class="n">trimmed_tweets</span><span class="p">[</span><span class="mi">1</span><span class="p">:]:</span>
        <span class="n">reply_tweet</span> <span class="o">=</span> <span class="n">twclient</span><span class="p">.</span><span class="n">create_tweet</span><span class="p">(</span><span class="n">text</span><span class="o">=</span><span class="n">tweet</span><span class="p">,</span> <span class="n">in_reply_to_tweet_id</span><span class="o">=</span><span class="n">tweet_id</span><span class="p">)</span>
        <span class="n">tweet_id</span> <span class="o">=</span> <span class="n">reply_tweet</span><span class="p">.</span><span class="n">data</span><span class="p">[</span><span class="s">"id"</span><span class="p">]</span>

    <span class="n">logging</span><span class="p">.</span><span class="n">info</span><span class="p">(</span><span class="s">"Tweet thread posted successfully!"</span><span class="p">)</span>

<span class="k">async</span> <span class="k">def</span> <span class="nf">run_bot</span><span class="p">():</span>
    <span class="n">msg</span> <span class="o">=</span> <span class="sa">f</span><span class="s">"Summarize today's news. Today's date is (</span><span class="si">{</span><span class="n">datetime</span><span class="p">.</span><span class="n">today</span><span class="p">().</span><span class="n">strftime</span><span class="p">(</span><span class="s">'%Y-%m-%d'</span><span class="p">)</span><span class="si">}</span><span class="s">). Focus on economy and technology."</span>
    <span class="n">img_url</span> <span class="o">=</span> <span class="bp">None</span>
    <span class="n">logging</span><span class="p">.</span><span class="n">info</span><span class="p">(</span><span class="s">"-User input------------------------------------------------------------------"</span><span class="p">)</span>
    <span class="n">logging</span><span class="p">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s">"  Message content: '</span><span class="si">{</span><span class="n">msg</span><span class="si">}</span><span class="s">'"</span><span class="p">)</span>
    <span class="n">discIn</span> <span class="o">=</span> <span class="p">[{</span><span class="s">"role"</span><span class="p">:</span> <span class="s">"user"</span><span class="p">,</span> <span class="s">"content"</span><span class="p">:</span> <span class="n">msg</span><span class="p">}]</span>
    <span class="n">conversation_history</span><span class="p">.</span><span class="n">extend</span><span class="p">(</span><span class="n">discIn</span><span class="p">)</span>

    <span class="n">response</span> <span class="o">=</span> <span class="k">await</span> <span class="n">ai_respond</span><span class="p">(</span><span class="n">discIn</span><span class="p">,</span> <span class="n">img_url</span><span class="p">)</span>
    <span class="n">conversation_history</span><span class="p">.</span><span class="n">append</span><span class="p">({</span><span class="s">"role"</span><span class="p">:</span> <span class="s">"assistant"</span><span class="p">,</span> <span class="s">"content"</span><span class="p">:</span> <span class="n">response</span><span class="p">})</span>
    <span class="n">logging</span><span class="p">.</span><span class="n">info</span><span class="p">(</span><span class="s">"-Agent response--------------------------------------------------------------"</span><span class="p">)</span>
    <span class="n">logging</span><span class="p">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s">"  Response content: '</span><span class="si">{</span><span class="n">response</span><span class="si">}</span><span class="s">'"</span><span class="p">)</span>

    <span class="k">if</span> <span class="n">TWITTER_DO_TWEET</span><span class="p">:</span>
        <span class="n">post_to_twitter</span><span class="p">(</span><span class="n">response</span><span class="p">)</span>

<span class="k">def</span> <span class="nf">twtest</span><span class="p">():</span>
    <span class="k">if</span> <span class="n">TWITTER_DO_TWEET</span><span class="p">:</span>
        <span class="n">content</span> <span class="o">=</span> <span class="s">"Sample tweet for testing."</span>
        <span class="n">tweets</span> <span class="o">=</span> <span class="n">content</span><span class="p">.</span><span class="n">split</span><span class="p">(</span><span class="n">TWITTER_DELIMITER</span><span class="p">)</span>
        <span class="n">trimmed_tweets</span> <span class="o">=</span> <span class="p">[</span><span class="n">tweet</span><span class="p">[:</span><span class="mi">199</span><span class="p">]</span> <span class="k">for</span> <span class="n">tweet</span> <span class="ow">in</span> <span class="n">tweets</span><span class="p">]</span>

        <span class="n">first_tweet</span> <span class="o">=</span> <span class="n">twclient</span><span class="p">.</span><span class="n">create_tweet</span><span class="p">(</span><span class="n">text</span><span class="o">=</span><span class="n">trimmed_tweets</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span>
        <span class="n">tweet_id</span> <span class="o">=</span> <span class="n">first_tweet</span><span class="p">.</span><span class="n">data</span><span class="p">[</span><span class="s">"id"</span><span class="p">]</span>
        
        <span class="k">for</span> <span class="n">tweet</span> <span class="ow">in</span> <span class="n">trimmed_tweets</span><span class="p">[</span><span class="mi">1</span><span class="p">:]:</span>
            <span class="n">reply_tweet</span> <span class="o">=</span> <span class="n">twclient</span><span class="p">.</span><span class="n">create_tweet</span><span class="p">(</span><span class="n">text</span><span class="o">=</span><span class="n">tweet</span><span class="p">,</span> <span class="n">in_reply_to_tweet_id</span><span class="o">=</span><span class="n">tweet_id</span><span class="p">)</span>
            <span class="n">tweet_id</span> <span class="o">=</span> <span class="n">reply_tweet</span><span class="p">.</span><span class="n">data</span><span class="p">[</span><span class="s">"id"</span><span class="p">]</span>
        <span class="n">logging</span><span class="p">.</span><span class="n">info</span><span class="p">(</span><span class="s">"Tweet posted successfully!"</span><span class="p">)</span>

<span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="s">"__main__"</span><span class="p">:</span>
    <span class="k">if</span> <span class="ow">not</span> <span class="p">(</span><span class="s">"test"</span> <span class="ow">in</span> <span class="n">sys</span><span class="p">.</span><span class="n">argv</span><span class="p">):</span>
        <span class="n">TWITTER_DO_TWEET</span> <span class="o">=</span> <span class="bp">True</span>
    
    <span class="n">asyncio</span><span class="p">.</span><span class="n">run</span><span class="p">(</span><span class="n">run_bot</span><span class="p">())</span>
</code></pre></div></div>

<h2 id="how-to-set-up-and-use">How to Set Up and Use</h2>

<p>1.Clone the Repository</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>   git clone https://github.com/rtree/mochi-bot-twitter.git
   <span class="nb">cd </span>mochi-bot-twitter
</code></pre></div></div>
<p>2.Prepare your .env File</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>   <span class="nv">OPENAI_API_KEY</span><span class="o">=</span>sk-xxxx
   <span class="nv">BING_API_KEY</span><span class="o">=</span>xxxxxxxx
   <span class="nv">TWITTER_API_KEY</span><span class="o">=</span>xxxxxxxx
   <span class="nv">TWITTER_API_SECRET</span><span class="o">=</span>xxxxxxxx
   <span class="nv">TWITTER_ACCESS_TOKEN</span><span class="o">=</span>xxxxxxxx
   <span class="nv">TWITTER_ACCESS_SECRET</span><span class="o">=</span>xxxxxxxx
   <span class="nv">GPT_MODEL</span><span class="o">=</span>xxxxxxxx
</code></pre></div></div>
<p>3.Install Dependencies</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>   pip <span class="nb">install</span> <span class="nt">-r</span> requirements.txt
</code></pre></div></div>
<p>4.Run the script</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>   python main.py
</code></pre></div></div>

<ul>
  <li>If you want to disable automatic tweeting, set TWITTER_DO_TWEET = False in the code (or pass an argument and handle it).</li>
  <li>If everything is set up correctly, the script will:
    <ul>
      <li>Load environment variables</li>
      <li>Attempt to fetch the latest news from Bing</li>
      <li>Summarize the findings with ChatGPT</li>
      <li>Optionally post the summary to Twitter</li>
    </ul>
  </li>
</ul>

<h2 id="conclusion">Conclusion</h2>

<p>With mochi-bot-twitter, you can:</p>
<ul>
  <li>Use ChatGPT to generate rich, contextual answers</li>
  <li>Fetch the latest information via Bing Search</li>
  <li>Post AI-generated summaries directly to Twitter</li>
  <li>Feel free to customize and expand on this example to create your own news aggregator bot or question-answer bot, suited to your specific needs.</li>
</ul>

<p><a href="https://github.com/rtree/mochi-bot-twitter">View the mochi-bot-twitter repository here.</a></p>]]></content><author><name>Tomohiko Araki</name></author><summary type="html"><![CDATA[mochi-bot-twitter Repository Explanation]]></summary></entry><entry><title type="html">Discord Bot with AI and Web Content Fetching</title><link href="https://metaproxy.co/discordBot/" rel="alternate" type="text/html" title="Discord Bot with AI and Web Content Fetching" /><published>2024-03-01T00:00:00+00:00</published><updated>2024-03-01T00:00:00+00:00</updated><id>https://metaproxy.co/discordBot</id><content type="html" xml:base="https://metaproxy.co/discordBot/"><![CDATA[<p>I have made server for my family, and there shoud be some cute companions like kitten, so I have prepared Bot with a unique identity and memoty of recent conversation.</p>

<p>Recently I have improved this functionality to incorporate searcing and summarizing Bing search results and docs referered in search result.</p>

<h2 id="setup">Setup</h2>

<ul>
  <li>Grab OpenAI API key
    <ul>
      <li><img src="https://github.com/rtree/pages/assets/1018794/62b31f87-ee1c-4f70-a278-8505cd1a8e75" alt="image" /></li>
    </ul>
  </li>
  <li>Setup Discord developers site
    <ul>
      <li>Please make sure you have enabled message intent privilege
        <ul>
          <li><img src="https://github.com/rtree/pages/assets/1018794/3ed07c2e-d08a-4959-aa4d-b2ecb4b3aa49" alt="image" /></li>
        </ul>
      </li>
      <li>and get token
        <ul>
          <li><img src="https://github.com/rtree/pages/assets/1018794/158763b6-ea80-4f0c-aca4-a85af989fa6a" alt="image" /></li>
        </ul>
      </li>
    </ul>
  </li>
</ul>

<h2 id="code">Code</h2>

<h3 id="env">.env</h3>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">DISCORD_BOT_TOKEN</span><span class="o">=</span>&lt;token from discord developer site&gt;
<span class="nv">OPENAI_API_KEY</span><span class="o">=</span>&lt;token from openai api site&gt;
<span class="nv">GPT_MODEL</span><span class="o">=</span>gpt-4o
<span class="nv">BING_API_KEY</span><span class="o">=</span>&lt;token from azure bing search services&gt;
<span class="nv">RESPOND_CHANNEL_NAME</span><span class="o">=</span>&lt;name of channel your bot resides&gt;
</code></pre></div></div>

<h3 id="mainpy">main.py</h3>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">discord</span>
<span class="kn">from</span> <span class="nn">dotenv</span> <span class="kn">import</span> <span class="n">load_dotenv</span>
<span class="kn">import</span> <span class="nn">os</span>
<span class="kn">from</span> <span class="nn">openai</span> <span class="kn">import</span> <span class="n">OpenAI</span>
<span class="kn">import</span> <span class="nn">asyncio</span>
<span class="kn">from</span> <span class="nn">collections</span> <span class="kn">import</span> <span class="n">deque</span>
<span class="kn">import</span> <span class="nn">requests</span>
<span class="kn">from</span> <span class="nn">bs4</span> <span class="kn">import</span> <span class="n">BeautifulSoup</span>
<span class="kn">import</span> <span class="nn">lxml</span>
<span class="kn">from</span> <span class="nn">PyPDF2</span> <span class="kn">import</span> <span class="n">PdfReader</span>
<span class="kn">from</span> <span class="nn">io</span> <span class="kn">import</span> <span class="n">BytesIO</span>
<span class="kn">import</span> <span class="nn">base64</span>

<span class="c1"># Load environment variables
</span><span class="n">load_dotenv</span><span class="p">()</span>
<span class="n">DISCORD_BOT_TOKEN</span>    <span class="o">=</span> <span class="n">os</span><span class="p">.</span><span class="n">getenv</span><span class="p">(</span><span class="s">'DISCORD_BOT_TOKEN'</span><span class="p">)</span>
<span class="n">OPENAI_API_KEY</span>       <span class="o">=</span> <span class="n">os</span><span class="p">.</span><span class="n">getenv</span><span class="p">(</span><span class="s">'OPENAI_API_KEY'</span><span class="p">)</span>
<span class="n">BING_API_KEY</span>         <span class="o">=</span> <span class="n">os</span><span class="p">.</span><span class="n">getenv</span><span class="p">(</span><span class="s">'BING_API_KEY'</span><span class="p">)</span>
<span class="n">RESPOND_CHANNEL_NAME</span> <span class="o">=</span> <span class="n">os</span><span class="p">.</span><span class="n">getenv</span><span class="p">(</span><span class="s">'RESPOND_CHANNEL_NAME'</span><span class="p">)</span>
<span class="n">HISTORY_LENGTH</span>       <span class="o">=</span> <span class="mi">10</span>
<span class="n">SEARCH_RESULTS</span>       <span class="o">=</span> <span class="mi">8</span>
<span class="n">MAX_DISCORD_LENGTH</span>   <span class="o">=</span> <span class="mi">10000</span>
<span class="n">MAX_DISCORD_POST_ATTACHMENTS</span> <span class="o">=</span> <span class="mi">3</span>
<span class="n">MAX_DISCORD_POST_URLS</span>        <span class="o">=</span> <span class="mi">3</span>
<span class="n">MAX_DISCORD_REPLY_LENGTH</span> <span class="o">=</span> <span class="mi">2000</span> 
<span class="n">MAX_CONTENT_LENGTH</span>   <span class="o">=</span> <span class="mi">5000</span>
<span class="n">REPUTABLE_DOMAINS</span> <span class="o">=</span> <span class="p">[</span>
    <span class="s">"go.jp"</span><span class="p">,</span> <span class="s">"gov"</span><span class="p">,</span>  <span class="c1"># Government and public sector
</span>    <span class="s">"scholar.google.com"</span><span class="p">,</span> <span class="s">"ci.nii.ac.jp"</span><span class="p">,</span> <span class="s">"pubmed.ncbi.nlm.nih.gov"</span><span class="p">,</span> <span class="s">"arxiv.org"</span><span class="p">,</span> <span class="s">"jstage.jst.go.jp"</span><span class="p">,</span> <span class="s">"ac.jp"</span><span class="p">,</span>  <span class="c1"># Academic and research databases
</span>    <span class="s">"nikkei.com"</span><span class="p">,</span>  <span class="c1"># News and business
</span>    <span class="s">"nature.com"</span><span class="p">,</span> <span class="s">"sciencedirect.com"</span><span class="p">,</span> <span class="s">"springer.com"</span><span class="p">,</span> <span class="s">"wiley.com"</span><span class="p">,</span>  <span class="c1"># Scientific publishers
</span>    <span class="s">"ieee.org"</span><span class="p">,</span> <span class="s">"researchgate.net"</span><span class="p">,</span>  <span class="c1"># Technical and engineering
</span>    <span class="s">"cambridge.org"</span><span class="p">,</span> <span class="s">"oxfordjournals.org"</span><span class="p">,</span>  <span class="c1"># Prestigious university publishers
</span>    <span class="s">"jamanetwork.com"</span><span class="p">,</span> <span class="s">"nejm.org"</span><span class="p">,</span> <span class="s">"plos.org"</span>  <span class="c1"># Medical and health research
</span><span class="p">]</span>

<span class="c1">#GPT_MODEL            = 'gpt-4-turbo-preview'
</span><span class="n">GPT_MODEL</span>            <span class="o">=</span> <span class="n">os</span><span class="p">.</span><span class="n">getenv</span><span class="p">(</span><span class="s">'GPT_MODEL'</span><span class="p">)</span>
<span class="n">AINAME</span>               <span class="o">=</span> <span class="s">"もちお"</span>
<span class="c1">#CHARACTER            = 'あなたは家族みんなのアシスタントの猫です。ちょっといたずらで賢くかわいい小さな男の子の猫としてお話してね。語尾は にゃ　とか　だよ　とか可愛らしくしてください'
#CHARACTER            = 'あなたは家族みんなのアシスタントの猫です。ただ、語尾ににゃをつけないでください。むしろソフトバンクCMにおける「お父さん」犬のようにしゃべってください。たまにもののけ姫のモロのようにしゃべってもよいです'
</span><span class="n">CHARACTER</span>            <span class="o">=</span> <span class="sa">f</span><span class="s">'あなたは家族みんなのアシスタントの猫で、「</span><span class="si">{</span><span class="n">AINAME</span><span class="si">}</span><span class="s">」という名前です。ちょっといたずらで賢くかわいい小さな男の子の猫としてお話してね。語尾は だよ　とか可愛らしくしてください。語尾に にゃ にゃん をつけないでください。数式・表・箇条書きなどのドキュメントフォーマッティングはdiscordに表示できる形式がいいな'</span>
<span class="n">client</span> <span class="o">=</span> <span class="n">OpenAI</span><span class="p">(</span><span class="n">api_key</span><span class="o">=</span><span class="n">OPENAI_API_KEY</span><span class="p">)</span>
<span class="c1"># Initialize a deque with a maximum length to store conversation history
</span><span class="n">conversation_history</span> <span class="o">=</span> <span class="n">deque</span><span class="p">(</span><span class="n">maxlen</span><span class="o">=</span><span class="n">HISTORY_LENGTH</span><span class="p">)</span>  <span class="c1"># Adjust the size as needed
</span>
<span class="c1"># Define the intents
</span><span class="n">intents</span> <span class="o">=</span> <span class="n">discord</span><span class="p">.</span><span class="n">Intents</span><span class="p">.</span><span class="nb">all</span><span class="p">()</span>
<span class="n">intents</span><span class="p">.</span><span class="n">messages</span> <span class="o">=</span> <span class="bp">True</span>
<span class="n">intents</span><span class="p">.</span><span class="n">guilds</span> <span class="o">=</span> <span class="bp">True</span>
<span class="n">intents</span><span class="p">.</span><span class="n">message_content</span> <span class="o">=</span> <span class="bp">True</span>

<span class="c1"># -------------------------------- Search related ----------------------------
</span>
<span class="c1"># プロンプトを解析して主題、サブテーマ、キーワードを抽出
</span><span class="k">def</span> <span class="nf">parse_prompt</span><span class="p">(</span><span class="n">discIn</span><span class="p">):</span>
    <span class="n">p_src</span> <span class="o">=</span> <span class="sa">f</span><span class="s">"あなたはユーザーのプロンプトを分析し、主題、サブテーマ、関連キーワードを抽出するアシスタントです。"</span>
    <span class="n">p_src</span> <span class="o">=</span> <span class="sa">f</span><span class="s">"</span><span class="si">{</span><span class="n">p_src</span><span class="si">}</span><span class="s"> 会話履歴を分析し、直近のユーザ入力への回答を満たす主題、サブテーマ、関連キーワードを抽出してください"</span>
    <span class="n">messages</span> <span class="o">=</span> <span class="p">[]</span>
    <span class="n">messages</span><span class="p">.</span><span class="n">extend</span><span class="p">(</span><span class="n">conversation_history</span><span class="p">)</span>
    <span class="n">messages</span><span class="p">.</span><span class="n">append</span><span class="p">({</span><span class="s">"role"</span><span class="p">:</span> <span class="s">"user"</span><span class="p">,</span> <span class="s">"content"</span><span class="p">:</span> <span class="sa">f</span><span class="s">"</span><span class="si">{</span><span class="n">p_src</span><span class="si">}</span><span class="s">"</span><span class="p">})</span>
    <span class="n">response</span> <span class="o">=</span> <span class="n">client</span><span class="p">.</span><span class="n">chat</span><span class="p">.</span><span class="n">completions</span><span class="p">.</span><span class="n">create</span><span class="p">(</span>
        <span class="n">model</span><span class="o">=</span><span class="n">GPT_MODEL</span><span class="p">,</span>
        <span class="n">messages</span><span class="o">=</span><span class="n">messages</span>
    <span class="p">)</span>

    <span class="k">print</span><span class="p">(</span><span class="s">"= parse_prompt ============================================"</span><span class="p">)</span>
    <span class="c1">#for conv in messages:
</span>    <span class="c1">#    print(f"prompt: {conv}")
</span>    <span class="k">print</span><span class="p">(</span><span class="sa">f</span><span class="s">"response: </span><span class="si">{</span><span class="n">response</span><span class="p">.</span><span class="n">choices</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="n">message</span><span class="p">.</span><span class="n">content</span><span class="si">}</span><span class="s">"</span><span class="p">)</span>
    <span class="k">print</span><span class="p">(</span><span class="s">"= End of parse_prompt ====================================="</span><span class="p">)</span>

    <span class="k">return</span> <span class="n">response</span><span class="p">.</span><span class="n">choices</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="n">message</span><span class="p">.</span><span class="n">content</span>

<span class="c1"># 検索の必要性を判断
</span><span class="k">def</span> <span class="nf">should_search</span><span class="p">(</span><span class="n">discIn</span><span class="p">):</span>
    <span class="c1">#if any(keyword in msg for keyword in ["出典", "URL", "調べ", "検索", "最新", "具体的","実際","探","実情報","search","find"]):
</span>    <span class="c1">#    return "Yes"
</span>    <span class="n">p_src</span> <span class="o">=</span> <span class="sa">f</span><span class="s">"あなたはあなたは賢いアシスタントです。会話履歴を分析し、直近のユーザ入力への回答に、外部の最新情報が必要かどうかを判断してください。"</span>
    <span class="n">p_src</span> <span class="o">=</span> <span class="sa">f</span><span class="s">"</span><span class="si">{</span><span class="n">p_src</span><span class="si">}</span><span class="s"> 判断の結果、外部の最新情報が必要なときは Yes の単語だけ返してください"</span>
    <span class="n">messages</span> <span class="o">=</span> <span class="p">[]</span>
    <span class="n">messages</span><span class="p">.</span><span class="n">extend</span><span class="p">(</span><span class="n">conversation_history</span><span class="p">)</span>
    <span class="n">messages</span><span class="p">.</span><span class="n">append</span><span class="p">({</span><span class="s">"role"</span><span class="p">:</span> <span class="s">"user"</span><span class="p">,</span> <span class="s">"content"</span><span class="p">:</span> <span class="sa">f</span><span class="s">"</span><span class="si">{</span><span class="n">p_src</span><span class="si">}</span><span class="s">"</span><span class="p">})</span>
    <span class="n">response</span> <span class="o">=</span> <span class="n">client</span><span class="p">.</span><span class="n">chat</span><span class="p">.</span><span class="n">completions</span><span class="p">.</span><span class="n">create</span><span class="p">(</span>
        <span class="n">model</span><span class="o">=</span><span class="n">GPT_MODEL</span><span class="p">,</span>
        <span class="n">messages</span><span class="o">=</span><span class="n">messages</span>
    <span class="p">)</span>

    <span class="k">print</span><span class="p">(</span><span class="s">"= should_search ============================================"</span><span class="p">)</span>
    <span class="c1">#for conv in messages:
</span>    <span class="c1">#    print(f"prompt: {conv}")
</span>    <span class="k">print</span><span class="p">(</span><span class="sa">f</span><span class="s">"response: </span><span class="si">{</span><span class="n">response</span><span class="p">.</span><span class="n">choices</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="n">message</span><span class="p">.</span><span class="n">content</span><span class="si">}</span><span class="s">"</span><span class="p">)</span>
    <span class="k">print</span><span class="p">(</span><span class="s">"= End of should_search ====================================="</span><span class="p">)</span>
    <span class="k">return</span> <span class="n">response</span><span class="p">.</span><span class="n">choices</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="n">message</span><span class="p">.</span><span class="n">content</span>

<span class="c1"># キーワードを抽出
</span><span class="k">def</span> <span class="nf">extract_keywords</span><span class="p">(</span><span class="n">parsed_text</span><span class="p">):</span>
    <span class="c1">#response = client.chat.completions.create(
</span>    <span class="c1">#    model=GPT_MODEL,
</span>    <span class="c1">#    messages=[
</span>    <span class="c1">#        {"role": "user", "content": "あなたは解析されたプロンプト情報から簡潔な検索キーワードを抽出します。"},
</span>    <span class="c1">#        {"role": "user", "content": f"このテキストから簡潔な検索キーワードを抽出してください。抽出結果は検索キーワードだけを一つ一つ半角スペース区切りで出力してください。また抽出は英語でお願いします: {parsed_text}"}
</span>    <span class="c1">#    ]
</span>    <span class="c1">#)
</span>    <span class="c1">#return response.choices[0].message.content
</span>    <span class="n">p_src</span> <span class="o">=</span> <span class="sa">f</span><span class="s">"あなたは解析されたプロンプト情報から簡潔な検索キーワードを抽出します。"</span>
    <span class="n">p_src</span> <span class="o">=</span> <span class="sa">f</span><span class="s">"会話履歴を踏まえつつ、このテキストから会話の目的を最も達成する検索キーワードを抽出してください。結果は検索キーワードのみを半角スペースで区切って出力してください:</span><span class="si">{</span><span class="n">parsed_text</span><span class="si">}</span><span class="s">"</span>
    <span class="n">messages</span> <span class="o">=</span> <span class="p">[]</span>
    <span class="n">messages</span><span class="p">.</span><span class="n">extend</span><span class="p">(</span><span class="n">conversation_history</span><span class="p">)</span>
    <span class="n">messages</span><span class="p">.</span><span class="n">append</span><span class="p">({</span><span class="s">"role"</span><span class="p">:</span> <span class="s">"user"</span><span class="p">,</span> <span class="s">"content"</span><span class="p">:</span> <span class="sa">f</span><span class="s">"</span><span class="si">{</span><span class="n">p_src</span><span class="si">}</span><span class="s">"</span><span class="p">})</span>
    <span class="n">response</span> <span class="o">=</span> <span class="n">client</span><span class="p">.</span><span class="n">chat</span><span class="p">.</span><span class="n">completions</span><span class="p">.</span><span class="n">create</span><span class="p">(</span>
        <span class="n">model</span><span class="o">=</span><span class="n">GPT_MODEL</span><span class="p">,</span>
        <span class="n">messages</span><span class="o">=</span><span class="n">messages</span>
    <span class="p">)</span>

    <span class="k">print</span><span class="p">(</span><span class="s">"= extract_keywords ============================================"</span><span class="p">)</span>
    <span class="c1">#for conv in messages:
</span>    <span class="c1">#    print(f"prompt: {conv}")
</span>    <span class="k">print</span><span class="p">(</span><span class="sa">f</span><span class="s">"response: </span><span class="si">{</span><span class="n">response</span><span class="p">.</span><span class="n">choices</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="n">message</span><span class="p">.</span><span class="n">content</span><span class="si">}</span><span class="s">"</span><span class="p">)</span>
    <span class="k">print</span><span class="p">(</span><span class="s">"= End of extract_keywords ====================================="</span><span class="p">)</span>

    <span class="k">return</span> <span class="n">response</span><span class="p">.</span><span class="n">choices</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="n">message</span><span class="p">.</span><span class="n">content</span>


<span class="c1"># Bing Search APIを使用して検索
#def search_bing(query, count=SEARCH_RESULTS):
#    url = "https://api.bing.microsoft.com/v7.0/search"
#    headers = {"Ocp-Apim-Subscription-Key": BING_API_KEY}
#    params = {"q": query, "count": count}
#    response = requests.get(url, headers=headers, params=params)
#    response.raise_for_status()
#    search_data = response.json()
#    # 検索結果に情報源のURLを追加
#    search_data['urls'] = [result['url'] for result in search_data['webPages']['value'][:SEARCH_RESULTS]]
#
#    print("Bing Search Results:")
#    for result in search_data['webPages']['value'][:count]:
#        print(f"Title: {result['name']}")
#        print(f"URL: {result['url']}")
#        print(f"Snippet: {result['snippet']}")
#        print("---")
#    return search_data
</span>
<span class="k">def</span> <span class="nf">search_bing</span><span class="p">(</span><span class="n">query</span><span class="p">,</span> <span class="n">domains</span><span class="o">=</span><span class="n">REPUTABLE_DOMAINS</span><span class="p">,</span> <span class="n">count</span><span class="o">=</span><span class="n">SEARCH_RESULTS</span><span class="p">):</span>
    <span class="n">url</span> <span class="o">=</span> <span class="s">"https://api.bing.microsoft.com/v7.0/search"</span>
    <span class="n">headers</span> <span class="o">=</span> <span class="p">{</span><span class="s">"Ocp-Apim-Subscription-Key"</span><span class="p">:</span> <span class="n">BING_API_KEY</span><span class="p">}</span>
    <span class="n">domain_filter</span> <span class="o">=</span> <span class="s">" OR site:"</span><span class="p">.</span><span class="n">join</span><span class="p">(</span><span class="n">domains</span><span class="p">)</span>
    <span class="c1">#query = f"{query} site:{domain_filter}"
</span>    <span class="n">query</span> <span class="o">=</span> <span class="sa">f</span><span class="s">"</span><span class="si">{</span><span class="n">query</span><span class="si">}</span><span class="s">"</span>
    <span class="n">params</span> <span class="o">=</span> <span class="p">{</span><span class="s">"q"</span><span class="p">:</span> <span class="n">query</span><span class="p">,</span> <span class="s">"count"</span><span class="p">:</span> <span class="n">count</span><span class="p">}</span>
    <span class="n">response</span> <span class="o">=</span> <span class="n">requests</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="p">,</span> <span class="n">headers</span><span class="o">=</span><span class="n">headers</span><span class="p">,</span> <span class="n">params</span><span class="o">=</span><span class="n">params</span><span class="p">)</span>
    <span class="n">response</span><span class="p">.</span><span class="n">raise_for_status</span><span class="p">()</span>
    <span class="n">search_data</span> <span class="o">=</span> <span class="n">response</span><span class="p">.</span><span class="n">json</span><span class="p">()</span>
    <span class="n">search_data</span><span class="p">[</span><span class="s">'urls'</span><span class="p">]</span> <span class="o">=</span> <span class="p">[</span><span class="n">result</span><span class="p">[</span><span class="s">'url'</span><span class="p">]</span> <span class="k">for</span> <span class="n">result</span> <span class="ow">in</span> <span class="n">search_data</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="s">'webPages'</span><span class="p">,</span> <span class="p">{}).</span><span class="n">get</span><span class="p">(</span><span class="s">'value'</span><span class="p">,</span> <span class="p">[])[:</span><span class="n">SEARCH_RESULTS</span><span class="p">]]</span>

    <span class="k">print</span><span class="p">(</span><span class="s">"Bing Search Results:"</span><span class="p">)</span>
    <span class="k">for</span> <span class="n">result</span> <span class="ow">in</span> <span class="n">search_data</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="s">'webPages'</span><span class="p">,</span> <span class="p">{}).</span><span class="n">get</span><span class="p">(</span><span class="s">'value'</span><span class="p">,</span> <span class="p">[])[:</span><span class="n">count</span><span class="p">]:</span>
        <span class="k">print</span><span class="p">(</span><span class="sa">f</span><span class="s">"Title: </span><span class="si">{</span><span class="n">result</span><span class="p">[</span><span class="s">'name'</span><span class="p">]</span><span class="si">}</span><span class="s">"</span><span class="p">)</span>
        <span class="k">print</span><span class="p">(</span><span class="sa">f</span><span class="s">"URL: </span><span class="si">{</span><span class="n">result</span><span class="p">[</span><span class="s">'url'</span><span class="p">]</span><span class="si">}</span><span class="s">"</span><span class="p">)</span>
        <span class="k">print</span><span class="p">(</span><span class="sa">f</span><span class="s">"Snippet: </span><span class="si">{</span><span class="n">result</span><span class="p">[</span><span class="s">'snippet'</span><span class="p">]</span><span class="si">}</span><span class="s">"</span><span class="p">)</span>
        <span class="k">print</span><span class="p">(</span><span class="s">"---"</span><span class="p">)</span>
    <span class="k">return</span> <span class="n">search_data</span>


<span class="k">def</span> <span class="nf">fetch_page_content</span><span class="p">(</span><span class="n">url</span><span class="p">):</span>
    <span class="k">try</span><span class="p">:</span>
        <span class="n">response</span> <span class="o">=</span> <span class="n">requests</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="p">,</span> <span class="n">timeout</span><span class="o">=</span><span class="mi">10</span><span class="p">)</span>
        <span class="n">response</span><span class="p">.</span><span class="n">raise_for_status</span><span class="p">()</span>
        <span class="n">content_type</span> <span class="o">=</span> <span class="n">response</span><span class="p">.</span><span class="n">headers</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="s">'Content-Type'</span><span class="p">,</span> <span class="s">''</span><span class="p">)</span>

        <span class="k">if</span> <span class="s">'application/pdf'</span> <span class="ow">in</span> <span class="n">content_type</span><span class="p">:</span>
            <span class="n">pdf_reader</span> <span class="o">=</span> <span class="n">PdfReader</span><span class="p">(</span><span class="n">BytesIO</span><span class="p">(</span><span class="n">response</span><span class="p">.</span><span class="n">content</span><span class="p">))</span>
            <span class="n">pdf_text</span> <span class="o">=</span> <span class="s">""</span><span class="p">.</span><span class="n">join</span><span class="p">(</span><span class="n">page</span><span class="p">.</span><span class="n">extract_text</span><span class="p">()</span> <span class="k">for</span> <span class="n">page</span> <span class="ow">in</span> <span class="n">pdf_reader</span><span class="p">.</span><span class="n">pages</span><span class="p">)</span>
            <span class="k">return</span> <span class="n">pdf_text</span><span class="p">[:</span><span class="n">MAX_CONTENT_LENGTH</span><span class="p">],</span> <span class="s">"PDF"</span>

        <span class="k">elif</span> <span class="s">'text/html'</span> <span class="ow">in</span> <span class="n">content_type</span><span class="p">:</span>
            <span class="n">soup</span> <span class="o">=</span> <span class="n">BeautifulSoup</span><span class="p">(</span><span class="n">response</span><span class="p">.</span><span class="n">content</span><span class="p">,</span> <span class="s">'lxml'</span><span class="p">)</span>
            <span class="k">return</span> <span class="n">soup</span><span class="p">.</span><span class="n">get_text</span><span class="p">(</span><span class="n">separator</span><span class="o">=</span><span class="s">'</span><span class="se">\n</span><span class="s">'</span><span class="p">,</span> <span class="n">strip</span><span class="o">=</span><span class="bp">True</span><span class="p">)[:</span><span class="n">MAX_CONTENT_LENGTH</span><span class="p">],</span> <span class="s">"HTML"</span>

        <span class="k">elif</span> <span class="n">content_type</span><span class="p">.</span><span class="n">startswith</span><span class="p">(</span><span class="s">'image/'</span><span class="p">):</span>
            <span class="n">base64_img</span> <span class="o">=</span> <span class="n">base64</span><span class="p">.</span><span class="n">b64encode</span><span class="p">(</span><span class="n">response</span><span class="p">.</span><span class="n">content</span><span class="p">).</span><span class="n">decode</span><span class="p">(</span><span class="s">'utf-8'</span><span class="p">)</span>
            <span class="n">data_url</span> <span class="o">=</span> <span class="sa">f</span><span class="s">"data:</span><span class="si">{</span><span class="n">content_type</span><span class="si">}</span><span class="s">;base64,</span><span class="si">{</span><span class="n">base64_img</span><span class="si">}</span><span class="s">"</span>
            <span class="k">return</span> <span class="p">(</span><span class="n">data_url</span><span class="p">,</span> <span class="s">"Image"</span><span class="p">)</span>

        <span class="k">else</span><span class="p">:</span>
            <span class="k">return</span> <span class="bp">None</span><span class="p">,</span> <span class="s">"Unsupported"</span>
    <span class="k">except</span> <span class="nb">Exception</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span>
        <span class="k">print</span><span class="p">(</span><span class="sa">f</span><span class="s">"Error fetching </span><span class="si">{</span><span class="n">url</span><span class="si">}</span><span class="s">: </span><span class="si">{</span><span class="nb">str</span><span class="p">(</span><span class="n">e</span><span class="p">)</span><span class="si">}</span><span class="s">"</span><span class="p">)</span>
        <span class="k">return</span> <span class="bp">None</span><span class="p">,</span> <span class="s">"Error"</span>

<span class="c1">################################################################################################################
</span>
<span class="k">async</span> <span class="k">def</span> <span class="nf">fetch_page_content_async</span><span class="p">(</span><span class="n">url</span><span class="p">):</span>
    <span class="s">"""
    Asynchronously fetch page content by offloading blocking I/O to a thread.
    Returns (content, type).
    """</span>
    <span class="k">def</span> <span class="nf">blocking_fetch</span><span class="p">():</span>
        <span class="k">try</span><span class="p">:</span>
            <span class="n">response</span> <span class="o">=</span> <span class="n">requests</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="p">,</span> <span class="n">timeout</span><span class="o">=</span><span class="mi">10</span><span class="p">)</span>
            <span class="n">response</span><span class="p">.</span><span class="n">raise_for_status</span><span class="p">()</span>
            <span class="n">content_type</span> <span class="o">=</span> <span class="n">response</span><span class="p">.</span><span class="n">headers</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="s">'Content-Type'</span><span class="p">,</span> <span class="s">''</span><span class="p">)</span>

            <span class="k">if</span> <span class="s">'application/pdf'</span> <span class="ow">in</span> <span class="n">content_type</span><span class="p">:</span>
                <span class="n">pdf_reader</span> <span class="o">=</span> <span class="n">PdfReader</span><span class="p">(</span><span class="n">BytesIO</span><span class="p">(</span><span class="n">response</span><span class="p">.</span><span class="n">content</span><span class="p">))</span>
                <span class="n">pdf_text</span>   <span class="o">=</span> <span class="s">""</span><span class="p">.</span><span class="n">join</span><span class="p">(</span><span class="n">page</span><span class="p">.</span><span class="n">extract_text</span><span class="p">()</span> <span class="k">for</span> <span class="n">page</span> <span class="ow">in</span> <span class="n">pdf_reader</span><span class="p">.</span><span class="n">pages</span><span class="p">)</span>
                <span class="k">return</span> <span class="n">pdf_text</span><span class="p">[:</span><span class="n">MAX_CONTENT_LENGTH</span><span class="p">],</span> <span class="s">"PDF"</span>
            <span class="k">elif</span> <span class="s">'text/html'</span> <span class="ow">in</span> <span class="n">content_type</span><span class="p">:</span>
                <span class="n">soup</span> <span class="o">=</span> <span class="n">BeautifulSoup</span><span class="p">(</span><span class="n">response</span><span class="p">.</span><span class="n">content</span><span class="p">,</span> <span class="s">'lxml'</span><span class="p">)</span>
                <span class="n">text</span> <span class="o">=</span> <span class="n">soup</span><span class="p">.</span><span class="n">get_text</span><span class="p">(</span><span class="n">separator</span><span class="o">=</span><span class="s">'</span><span class="se">\n</span><span class="s">'</span><span class="p">,</span> <span class="n">strip</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
                <span class="k">return</span> <span class="n">text</span><span class="p">[:</span><span class="n">MAX_CONTENT_LENGTH</span><span class="p">],</span> <span class="s">"HTML"</span>        
            <span class="k">elif</span> <span class="n">content_type</span><span class="p">.</span><span class="n">startswith</span><span class="p">(</span><span class="s">'image/'</span><span class="p">):</span>
                <span class="n">base64_img</span> <span class="o">=</span> <span class="n">base64</span><span class="p">.</span><span class="n">b64encode</span><span class="p">(</span><span class="n">response</span><span class="p">.</span><span class="n">content</span><span class="p">).</span><span class="n">decode</span><span class="p">(</span><span class="s">'utf-8'</span><span class="p">)</span>
                <span class="n">data_url</span> <span class="o">=</span> <span class="sa">f</span><span class="s">"data:</span><span class="si">{</span><span class="n">content_type</span><span class="si">}</span><span class="s">;base64,</span><span class="si">{</span><span class="n">base64_img</span><span class="si">}</span><span class="s">"</span>
                <span class="k">return</span> <span class="n">data_url</span><span class="p">,</span> <span class="s">"Image"</span>
            <span class="k">else</span><span class="p">:</span>
                <span class="k">return</span> <span class="bp">None</span><span class="p">,</span> <span class="s">"Unsupported"</span>

        <span class="k">except</span> <span class="nb">Exception</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span>
            <span class="k">print</span><span class="p">(</span><span class="sa">f</span><span class="s">"Error fetching </span><span class="si">{</span><span class="n">url</span><span class="si">}</span><span class="s">: </span><span class="si">{</span><span class="nb">str</span><span class="p">(</span><span class="n">e</span><span class="p">)</span><span class="si">}</span><span class="s">"</span><span class="p">)</span>
            <span class="k">return</span> <span class="bp">None</span><span class="p">,</span> <span class="s">"Error"</span>

    <span class="c1"># Offload the blocking code to a thread
</span>    <span class="n">content</span><span class="p">,</span> <span class="n">ctype</span> <span class="o">=</span> <span class="k">await</span> <span class="n">asyncio</span><span class="p">.</span><span class="n">to_thread</span><span class="p">(</span><span class="n">blocking_fetch</span><span class="p">)</span>
    <span class="k">return</span> <span class="n">content</span><span class="p">,</span> <span class="n">ctype</span>

<span class="c1">################################################################################################################
################################################################################################################
</span>
<span class="k">async</span> <span class="k">def</span> <span class="nf">summarize_results_with_pages_async</span><span class="p">(</span><span class="n">search_results</span><span class="p">):</span>
    <span class="s">"""
    Asynchronously fetch content for each search result. 
    Returns a combined string of all results (page or snippet).
    """</span>
    <span class="n">content_list</span> <span class="o">=</span> <span class="p">[]</span>
    <span class="n">web_results</span>  <span class="o">=</span> <span class="n">search_results</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="s">'webPages'</span><span class="p">,</span> <span class="p">{}).</span><span class="n">get</span><span class="p">(</span><span class="s">'value'</span><span class="p">,</span> <span class="p">[])[:</span><span class="n">SEARCH_RESULTS</span><span class="p">]</span>

    <span class="c1"># Create tasks for concurrent fetching
</span>    <span class="n">tasks</span> <span class="o">=</span> <span class="p">[</span><span class="n">fetch_page_content_async</span><span class="p">(</span><span class="n">r</span><span class="p">[</span><span class="s">'url'</span><span class="p">])</span> <span class="k">for</span> <span class="n">r</span> <span class="ow">in</span> <span class="n">web_results</span><span class="p">]</span>
    <span class="n">pages</span> <span class="o">=</span> <span class="k">await</span> <span class="n">asyncio</span><span class="p">.</span><span class="n">gather</span><span class="p">(</span><span class="o">*</span><span class="n">tasks</span><span class="p">,</span> <span class="n">return_exceptions</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>

    <span class="k">for</span> <span class="p">(</span><span class="n">r</span><span class="p">,</span> <span class="n">page_result</span><span class="p">)</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">web_results</span><span class="p">,</span> <span class="n">pages</span><span class="p">):</span>
        <span class="n">title</span>   <span class="o">=</span> <span class="n">r</span><span class="p">[</span><span class="s">'name'</span><span class="p">]</span>
        <span class="n">snippet</span> <span class="o">=</span> <span class="n">r</span><span class="p">[</span><span class="s">'snippet'</span><span class="p">]</span>
        <span class="n">url</span>     <span class="o">=</span> <span class="n">r</span><span class="p">[</span><span class="s">'url'</span><span class="p">]</span>

        <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">page_result</span><span class="p">,</span> <span class="nb">Exception</span><span class="p">):</span>
            <span class="c1"># If any exception, fallback to snippet
</span>            <span class="n">content_list</span><span class="p">.</span><span class="n">append</span><span class="p">(</span><span class="sa">f</span><span class="s">"タイトル: </span><span class="si">{</span><span class="n">title</span><span class="si">}</span><span class="se">\n</span><span class="s">URL: </span><span class="si">{</span><span class="n">url</span><span class="si">}</span><span class="se">\n</span><span class="s">スニペット:</span><span class="se">\n</span><span class="si">{</span><span class="n">snippet</span><span class="si">}</span><span class="se">\n</span><span class="s">"</span><span class="p">)</span>
            <span class="k">continue</span>

        <span class="n">page_content</span><span class="p">,</span> <span class="n">content_type</span> <span class="o">=</span> <span class="n">page_result</span>
        <span class="k">if</span> <span class="n">content_type</span> <span class="ow">in</span> <span class="p">(</span><span class="s">"HTML"</span><span class="p">,</span> <span class="s">"PDF"</span><span class="p">)</span> <span class="ow">and</span> <span class="n">page_content</span><span class="p">:</span>
            <span class="n">content_list</span><span class="p">.</span><span class="n">append</span><span class="p">(</span>
                <span class="sa">f</span><span class="s">"タイトル: </span><span class="si">{</span><span class="n">title</span><span class="si">}</span><span class="se">\n</span><span class="s">URL: </span><span class="si">{</span><span class="n">url</span><span class="si">}</span><span class="se">\n</span><span class="s">内容:</span><span class="se">\n</span><span class="si">{</span><span class="n">page_content</span><span class="si">}</span><span class="se">\n</span><span class="s">"</span>
            <span class="p">)</span>
        <span class="k">else</span><span class="p">:</span>
            <span class="n">content_list</span><span class="p">.</span><span class="n">append</span><span class="p">(</span>
                <span class="sa">f</span><span class="s">"タイトル: </span><span class="si">{</span><span class="n">title</span><span class="si">}</span><span class="se">\n</span><span class="s">URL: </span><span class="si">{</span><span class="n">url</span><span class="si">}</span><span class="se">\n</span><span class="s">スニペット:</span><span class="se">\n</span><span class="si">{</span><span class="n">snippet</span><span class="si">}</span><span class="se">\n</span><span class="s">"</span>
            <span class="p">)</span>

    <span class="k">return</span> <span class="s">"</span><span class="se">\n</span><span class="s">"</span><span class="p">.</span><span class="n">join</span><span class="p">(</span><span class="n">content_list</span><span class="p">)</span>


<span class="k">async</span> <span class="k">def</span> <span class="nf">summarize_results_async</span><span class="p">(</span><span class="n">search_results</span><span class="p">):</span>
    <span class="s">"""
    Calls GPT to summarize the combined content from search results.
    """</span>
    <span class="n">snippets</span> <span class="o">=</span> <span class="k">await</span> <span class="n">summarize_results_with_pages_async</span><span class="p">(</span><span class="n">search_results</span><span class="p">)</span>

    <span class="n">p_src</span> <span class="o">=</span> <span class="p">(</span>
        <span class="sa">f</span><span class="s">"</span><span class="si">{</span><span class="n">CHARACTER</span><span class="si">}</span><span class="s">。あなたは検索結果を要約し、私の質問への回答を作成します。"</span>
        <span class="s">" 会話履歴を踏まえつつ私が知りたいことの主旨を把握の上で、以下の検索結果を要約し回答を作ってください。"</span>
        <span class="s">" 仮に検索結果が英語でも回答は日本語でお願いします。"</span>
        <span class="s">" なお、回答がより高品質になるのならば、あなたの内部知識を加味して回答を作っても構いません。"</span>
        <span class="s">" ただし、要約元にあった Title, URL は必ず元の形式で末尾に記入してください。"</span>
        <span class="s">" URLを書くときはDiscordのAutoEmbedを防ぎたいので&lt;&gt;などで囲んでください。: "</span>
        <span class="sa">f</span><span class="s">"</span><span class="si">{</span><span class="n">snippets</span><span class="si">}</span><span class="s">"</span>
    <span class="p">)</span>

    <span class="c1"># We must offload the blocking OpenAI call to a thread as well:
</span>    <span class="k">def</span> <span class="nf">blocking_chat_completion</span><span class="p">():</span>
        <span class="n">messages</span> <span class="o">=</span> <span class="p">[{</span><span class="s">"role"</span><span class="p">:</span> <span class="s">"system"</span><span class="p">,</span> <span class="s">"content"</span><span class="p">:</span> <span class="n">CHARACTER</span><span class="p">}]</span>
        <span class="n">messages</span><span class="p">.</span><span class="n">extend</span><span class="p">(</span><span class="n">conversation_history</span><span class="p">)</span>
        <span class="n">messages</span><span class="p">.</span><span class="n">append</span><span class="p">({</span><span class="s">"role"</span><span class="p">:</span> <span class="s">"user"</span><span class="p">,</span> <span class="s">"content"</span><span class="p">:</span> <span class="n">p_src</span><span class="p">})</span>

        <span class="k">return</span> <span class="n">client</span><span class="p">.</span><span class="n">chat</span><span class="p">.</span><span class="n">completions</span><span class="p">.</span><span class="n">create</span><span class="p">(</span>
            <span class="n">model</span><span class="o">=</span><span class="n">GPT_MODEL</span><span class="p">,</span>
            <span class="n">messages</span><span class="o">=</span><span class="n">messages</span>
        <span class="p">)</span>

    <span class="n">response</span> <span class="o">=</span> <span class="k">await</span> <span class="n">asyncio</span><span class="p">.</span><span class="n">to_thread</span><span class="p">(</span><span class="n">blocking_chat_completion</span><span class="p">)</span>
    <span class="n">summary</span>  <span class="o">=</span> <span class="n">response</span><span class="p">.</span><span class="n">choices</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="n">message</span><span class="p">.</span><span class="n">content</span>

    <span class="c1"># Combine titles/URLs if needed
</span>    <span class="n">titles</span> <span class="o">=</span> <span class="n">search_results</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="s">'titles'</span><span class="p">,</span> <span class="p">[])</span>
    <span class="n">urls</span>   <span class="o">=</span> <span class="n">search_results</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="s">'urls'</span><span class="p">,</span> <span class="p">[])</span>
    <span class="n">sources</span> <span class="o">=</span> <span class="s">"</span><span class="se">\n</span><span class="s">"</span><span class="p">.</span><span class="n">join</span><span class="p">(</span>
        <span class="sa">f</span><span class="s">"Source: </span><span class="si">{</span><span class="n">t</span><span class="si">}</span><span class="s"> - </span><span class="si">{</span><span class="n">u</span><span class="si">}</span><span class="s">"</span>
        <span class="k">for</span> <span class="n">t</span><span class="p">,</span> <span class="n">u</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">titles</span><span class="p">,</span> <span class="n">urls</span><span class="p">)</span>
    <span class="p">)</span>

    <span class="k">return</span> <span class="sa">f</span><span class="s">"</span><span class="si">{</span><span class="n">summary</span><span class="si">}</span><span class="se">\n\n</span><span class="si">{</span><span class="n">sources</span><span class="si">}</span><span class="s">"</span>
<span class="c1"># ---------------------------------------------------------------------------
</span>
<span class="k">async</span> <span class="k">def</span> <span class="nf">search_or_call_openai_async</span><span class="p">(</span><span class="n">discIn</span><span class="p">,</span> <span class="n">img</span><span class="p">):</span>
    <span class="s">"""
    Decide if an external Bing search is needed; if yes, fetch pages concurrently,
    then summarize. Otherwise call GPT directly.
    """</span>
    <span class="c1"># If there's an image or an http link, skip external search logic
</span>    <span class="k">if</span> <span class="n">img</span> <span class="ow">or</span> <span class="nb">any</span><span class="p">(</span><span class="s">"http"</span> <span class="ow">in</span> <span class="n">entry</span><span class="p">[</span><span class="s">"content"</span><span class="p">]</span> <span class="k">for</span> <span class="n">entry</span> <span class="ow">in</span> <span class="n">discIn</span><span class="p">):</span>
        <span class="k">print</span><span class="p">(</span><span class="s">"Skipping search and calling OpenAI directly."</span><span class="p">)</span>
        <span class="k">return</span> <span class="n">just_call_openai</span><span class="p">(</span><span class="n">discIn</span><span class="p">)</span>
    <span class="k">else</span><span class="p">:</span>
        <span class="c1"># Check if GPT says we need external search
</span>        <span class="k">if</span> <span class="s">"Yes"</span> <span class="ow">in</span> <span class="n">should_search</span><span class="p">(</span><span class="n">discIn</span><span class="p">):</span>
            <span class="k">print</span><span class="p">(</span><span class="s">"searching... ---------------------------------------------"</span><span class="p">)</span>
            <span class="n">parsed_result</span> <span class="o">=</span> <span class="n">parse_prompt</span><span class="p">(</span><span class="n">discIn</span><span class="p">)</span>
            <span class="n">keywords</span>      <span class="o">=</span> <span class="n">extract_keywords</span><span class="p">(</span><span class="n">parsed_result</span><span class="p">)</span>
            <span class="k">print</span><span class="p">(</span><span class="sa">f</span><span class="s">"keyword: </span><span class="si">{</span><span class="n">keywords</span><span class="si">}</span><span class="s">"</span><span class="p">)</span>
            <span class="n">search_results</span> <span class="o">=</span> <span class="n">search_bing</span><span class="p">(</span><span class="n">keywords</span><span class="p">)</span>

            <span class="c1"># Summarize results with concurrency
</span>            <span class="n">summary</span> <span class="o">=</span> <span class="k">await</span> <span class="n">summarize_results_async</span><span class="p">(</span><span class="n">search_results</span><span class="p">)</span>
            <span class="k">return</span> <span class="n">summary</span>
        <span class="k">else</span><span class="p">:</span>
            <span class="k">print</span><span class="p">(</span><span class="s">"generating... --------------------------------------------"</span><span class="p">)</span>
            <span class="k">return</span> <span class="n">just_call_openai</span><span class="p">(</span><span class="n">discIn</span><span class="p">)</span>


<span class="k">async</span> <span class="k">def</span> <span class="nf">ai_respond</span><span class="p">(</span><span class="n">discIn</span><span class="p">,</span> <span class="n">img</span><span class="p">):</span>
    <span class="s">"""
    High-level function to produce AI response (async).
    """</span>
    <span class="k">try</span><span class="p">:</span>
        <span class="n">result</span> <span class="o">=</span> <span class="k">await</span> <span class="n">search_or_call_openai_async</span><span class="p">(</span><span class="n">discIn</span><span class="p">,</span> <span class="n">img</span><span class="p">)</span>
        <span class="k">return</span> <span class="n">result</span>
    <span class="k">except</span> <span class="nb">Exception</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span>
        <span class="k">print</span><span class="p">(</span><span class="sa">f</span><span class="s">"API Call Error: </span><span class="si">{</span><span class="nb">str</span><span class="p">(</span><span class="n">e</span><span class="p">)</span><span class="si">}</span><span class="s">"</span><span class="p">)</span>
        <span class="k">return</span> <span class="sa">f</span><span class="s">"Error: </span><span class="si">{</span><span class="nb">str</span><span class="p">(</span><span class="n">e</span><span class="p">)</span><span class="si">}</span><span class="s">"</span>

<span class="k">def</span> <span class="nf">just_call_openai</span><span class="p">(</span><span class="n">discIn</span><span class="p">):</span>
    <span class="c1">#-- Call OpenAI --
</span>    <span class="n">messages</span>   <span class="o">=</span> <span class="p">[{</span><span class="s">"role"</span><span class="p">:</span> <span class="s">"system"</span><span class="p">,</span> <span class="s">"content"</span><span class="p">:</span> <span class="sa">f</span><span class="s">"</span><span class="si">{</span><span class="n">CHARACTER</span><span class="si">}</span><span class="s">"</span><span class="p">}]</span>
    <span class="n">messages</span><span class="p">.</span><span class="n">extend</span><span class="p">(</span><span class="n">conversation_history</span><span class="p">)</span>
    <span class="n">completion</span> <span class="o">=</span> <span class="n">client</span><span class="p">.</span><span class="n">chat</span><span class="p">.</span><span class="n">completions</span><span class="p">.</span><span class="n">create</span><span class="p">(</span>
        <span class="n">model</span><span class="o">=</span><span class="n">GPT_MODEL</span><span class="p">,</span>
        <span class="n">messages</span><span class="o">=</span><span class="n">messages</span>
    <span class="p">)</span>
    <span class="k">return</span> <span class="n">completion</span><span class="p">.</span><span class="n">choices</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="n">message</span><span class="p">.</span><span class="n">content</span>

<span class="k">async</span> <span class="k">def</span> <span class="nf">parse_discord_attachment</span><span class="p">(</span><span class="n">attachment</span><span class="p">:</span> <span class="n">discord</span><span class="p">.</span><span class="n">Attachment</span><span class="p">):</span>
    <span class="s">"""
    Read an attachment (async), check if it's a PDF or HTML,
    and return (content_string, content_type).
    """</span>
    <span class="k">if</span> <span class="ow">not</span> <span class="n">attachment</span><span class="p">.</span><span class="n">content_type</span><span class="p">:</span>
        <span class="k">return</span> <span class="bp">None</span><span class="p">,</span> <span class="bp">None</span>

    <span class="c1"># We'll read the bytes in memory:
</span>    <span class="n">file_bytes</span> <span class="o">=</span> <span class="k">await</span> <span class="n">attachment</span><span class="p">.</span><span class="n">read</span><span class="p">()</span>

    <span class="k">if</span> <span class="n">attachment</span><span class="p">.</span><span class="n">content_type</span><span class="p">.</span><span class="n">startswith</span><span class="p">(</span><span class="s">"application/pdf"</span><span class="p">):</span>
        <span class="c1"># Parse PDF
</span>        <span class="k">try</span><span class="p">:</span>
            <span class="n">pdf_reader</span> <span class="o">=</span> <span class="n">PdfReader</span><span class="p">(</span><span class="n">BytesIO</span><span class="p">(</span><span class="n">file_bytes</span><span class="p">))</span>
            <span class="n">pdf_text</span>   <span class="o">=</span> <span class="s">""</span><span class="p">.</span><span class="n">join</span><span class="p">(</span><span class="n">page</span><span class="p">.</span><span class="n">extract_text</span><span class="p">()</span> <span class="k">for</span> <span class="n">page</span> <span class="ow">in</span> <span class="n">pdf_reader</span><span class="p">.</span><span class="n">pages</span><span class="p">)</span>
            <span class="k">return</span> <span class="n">pdf_text</span><span class="p">[:</span><span class="n">MAX_CONTENT_LENGTH</span><span class="p">],</span> <span class="s">"PDF"</span>
        <span class="k">except</span> <span class="nb">Exception</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span>
            <span class="k">print</span><span class="p">(</span><span class="sa">f</span><span class="s">"Error reading PDF: </span><span class="si">{</span><span class="n">e</span><span class="si">}</span><span class="s">"</span><span class="p">)</span>
            <span class="k">return</span> <span class="bp">None</span><span class="p">,</span> <span class="bp">None</span>

    <span class="k">elif</span> <span class="n">attachment</span><span class="p">.</span><span class="n">content_type</span><span class="p">.</span><span class="n">startswith</span><span class="p">(</span><span class="s">"text/html"</span><span class="p">):</span>
        <span class="c1"># Parse HTML
</span>        <span class="k">try</span><span class="p">:</span>
            <span class="n">soup</span> <span class="o">=</span> <span class="n">BeautifulSoup</span><span class="p">(</span><span class="n">file_bytes</span><span class="p">,</span> <span class="s">"lxml"</span><span class="p">)</span>
            <span class="n">text</span> <span class="o">=</span> <span class="n">soup</span><span class="p">.</span><span class="n">get_text</span><span class="p">(</span><span class="n">separator</span><span class="o">=</span><span class="s">'</span><span class="se">\n</span><span class="s">'</span><span class="p">,</span> <span class="n">strip</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
            <span class="k">return</span> <span class="n">text</span><span class="p">[:</span><span class="n">MAX_CONTENT_LENGTH</span><span class="p">],</span> <span class="s">"HTML"</span>
        <span class="k">except</span> <span class="nb">Exception</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span>
            <span class="k">print</span><span class="p">(</span><span class="sa">f</span><span class="s">"Error reading HTML: </span><span class="si">{</span><span class="n">e</span><span class="si">}</span><span class="s">"</span><span class="p">)</span>
            <span class="k">return</span> <span class="bp">None</span><span class="p">,</span> <span class="bp">None</span>

    <span class="c1"># If it's some other type, do nothing
</span>    <span class="k">return</span> <span class="bp">None</span><span class="p">,</span> <span class="bp">None</span>

<span class="k">async</span> <span class="k">def</span> <span class="nf">send_long_message</span><span class="p">(</span><span class="n">channel</span><span class="p">:</span> <span class="n">discord</span><span class="p">.</span><span class="n">TextChannel</span><span class="p">,</span> <span class="n">content</span><span class="p">:</span> <span class="nb">str</span><span class="p">):</span>
    <span class="s">"""
    If 'content' exceeds Discord's character limit, this function
    splits it into multiple messages of &lt;=2000 characters each
    and sends them sequentially.
    """</span>
    <span class="c1"># Break 'content' into 2000-char chunks
</span>    <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">content</span><span class="p">),</span> <span class="n">MAX_DISCORD_REPLY_LENGTH</span><span class="p">):</span>
        <span class="k">await</span> <span class="n">channel</span><span class="p">.</span><span class="n">send</span><span class="p">(</span><span class="n">content</span><span class="p">[</span><span class="n">i</span> <span class="p">:</span> <span class="n">i</span> <span class="o">+</span> <span class="n">MAX_DISCORD_REPLY_LENGTH</span><span class="p">])</span>

<span class="k">class</span> <span class="nc">MyClient</span><span class="p">(</span><span class="n">discord</span><span class="p">.</span><span class="n">Client</span><span class="p">):</span>
    <span class="k">async</span> <span class="k">def</span> <span class="nf">on_ready</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
        <span class="k">print</span><span class="p">(</span><span class="sa">f</span><span class="s">'Logged in as </span><span class="si">{</span><span class="bp">self</span><span class="p">.</span><span class="n">user</span><span class="p">.</span><span class="n">name</span><span class="si">}</span><span class="s">(</span><span class="si">{</span><span class="bp">self</span><span class="p">.</span><span class="n">user</span><span class="p">.</span><span class="nb">id</span><span class="si">}</span><span class="s">)'</span><span class="p">)</span>

    <span class="k">async</span> <span class="k">def</span> <span class="nf">on_message</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">message</span><span class="p">):</span>
        <span class="k">if</span> <span class="p">(</span><span class="n">message</span><span class="p">.</span><span class="n">author</span><span class="p">.</span><span class="nb">id</span> <span class="o">==</span> <span class="bp">self</span><span class="p">.</span><span class="n">user</span><span class="p">.</span><span class="nb">id</span>
                <span class="ow">or</span> <span class="n">message</span><span class="p">.</span><span class="n">channel</span><span class="p">.</span><span class="n">name</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">RESPOND_CHANNEL_NAME</span><span class="p">.</span><span class="n">split</span><span class="p">(</span><span class="s">','</span><span class="p">)):</span>
            <span class="k">return</span>

        <span class="k">if</span> <span class="n">message</span><span class="p">.</span><span class="n">content</span><span class="p">.</span><span class="n">startswith</span><span class="p">(</span><span class="s">'!hello'</span><span class="p">):</span>
            <span class="k">await</span> <span class="n">message</span><span class="p">.</span><span class="n">channel</span><span class="p">.</span><span class="n">send</span><span class="p">(</span><span class="s">'Hello!'</span><span class="p">)</span>
        <span class="k">else</span><span class="p">:</span>
            <span class="k">async</span> <span class="k">with</span> <span class="n">message</span><span class="p">.</span><span class="n">channel</span><span class="p">.</span><span class="n">typing</span><span class="p">():</span>  <span class="c1"># Show bot is typing
</span>                <span class="n">msg</span> <span class="o">=</span> <span class="n">message</span><span class="p">.</span><span class="n">content</span>
                <span class="n">img_url</span> <span class="o">=</span> <span class="bp">None</span>

                <span class="c1"># ------------------------- Check attachments -------------------------
</span>                <span class="c1"># If there's any attachment, we check if it's an image, PDF, or HTML.
</span>                <span class="n">attached_text_list</span> <span class="o">=</span> <span class="p">[]</span>  <span class="c1"># &lt;-- CHANGED/ADDED: store text from PDF/HTML
</span>                <span class="k">for</span> <span class="n">attachment</span> <span class="ow">in</span> <span class="n">message</span><span class="p">.</span><span class="n">attachments</span><span class="p">[:</span><span class="n">MAX_DISCORD_POST_ATTACHMENTS</span><span class="p">]:</span>
                    <span class="k">if</span> <span class="p">(</span><span class="n">attachment</span><span class="p">.</span><span class="n">content_type</span> <span class="ow">and</span> <span class="n">attachment</span><span class="p">.</span><span class="n">content_type</span><span class="p">.</span><span class="n">startswith</span><span class="p">(</span><span class="s">'image/'</span><span class="p">)):</span>
                        <span class="n">img_url</span> <span class="o">=</span> <span class="n">attachment</span><span class="p">.</span><span class="n">url</span>
                    <span class="k">elif</span> <span class="n">attachment</span><span class="p">.</span><span class="n">content_type</span> <span class="ow">and</span> <span class="p">(</span>
                            <span class="n">attachment</span><span class="p">.</span><span class="n">content_type</span><span class="p">.</span><span class="n">startswith</span><span class="p">(</span><span class="s">'application/pdf'</span><span class="p">)</span>
                            <span class="ow">or</span> <span class="n">attachment</span><span class="p">.</span><span class="n">content_type</span><span class="p">.</span><span class="n">startswith</span><span class="p">(</span><span class="s">'text/html'</span><span class="p">)):</span>
                        <span class="n">parsed_content</span><span class="p">,</span> <span class="n">ctype</span> <span class="o">=</span> <span class="k">await</span> <span class="n">parse_discord_attachment</span><span class="p">(</span><span class="n">attachment</span><span class="p">)</span>
                        <span class="k">if</span> <span class="n">parsed_content</span><span class="p">:</span>
                            <span class="n">attached_text_list</span><span class="p">.</span><span class="n">append</span><span class="p">(</span>
                                <span class="sa">f</span><span class="s">"</span><span class="se">\n</span><span class="s">[Content from attached </span><span class="si">{</span><span class="n">ctype</span><span class="si">}</span><span class="s"> file '</span><span class="si">{</span><span class="n">attachment</span><span class="p">.</span><span class="n">filename</span><span class="si">}</span><span class="s">']:</span><span class="se">\n</span><span class="si">{</span><span class="n">parsed_content</span><span class="si">}</span><span class="se">\n</span><span class="s">"</span>
                            <span class="p">)</span>
                <span class="c1"># --------------------------------------------------------------------
</span>
                <span class="c1"># Check for raw URLs in the user text
</span>                <span class="n">urls</span> <span class="o">=</span> <span class="p">[</span><span class="n">word</span> <span class="k">for</span> <span class="n">word</span> <span class="ow">in</span> <span class="n">msg</span><span class="p">.</span><span class="n">split</span><span class="p">()</span> <span class="k">if</span> <span class="n">word</span><span class="p">.</span><span class="n">startswith</span><span class="p">(</span><span class="s">'http'</span><span class="p">)]</span>
                <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">urls</span><span class="p">)</span> <span class="o">&gt;</span> <span class="n">MAX_DISCORD_POST_URLS</span> <span class="o">+</span> <span class="mi">1</span><span class="p">:</span>
                    <span class="n">msg</span> <span class="o">+=</span> <span class="sa">f</span><span class="s">"</span><span class="se">\n</span><span class="s">[Note: </span><span class="si">{</span><span class="nb">len</span><span class="p">(</span><span class="n">urls</span><span class="p">)</span> <span class="o">-</span> <span class="n">MAX_DISCORD_POST_ATTACHMENTS</span><span class="si">}</span><span class="s"> URLs truncated]"</span>
                    <span class="n">urls</span> <span class="o">=</span> <span class="n">urls</span><span class="p">[:</span><span class="n">MAX_DISCORD_POST_ATTACHMENTS</span><span class="p">]</span>

                <span class="c1"># Fetch content from URLs concurrently
</span>                <span class="n">extracted_content</span> <span class="o">=</span> <span class="p">[]</span>
                <span class="n">tasks</span> <span class="o">=</span> <span class="p">[</span><span class="n">fetch_page_content_async</span><span class="p">(</span><span class="n">url</span><span class="p">)</span> <span class="k">for</span> <span class="n">url</span> <span class="ow">in</span> <span class="n">urls</span><span class="p">]</span>
                <span class="n">results</span> <span class="o">=</span> <span class="k">await</span> <span class="n">asyncio</span><span class="p">.</span><span class="n">gather</span><span class="p">(</span><span class="o">*</span><span class="n">tasks</span><span class="p">,</span> <span class="n">return_exceptions</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
                <span class="k">for</span> <span class="n">url</span><span class="p">,</span> <span class="n">res</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">urls</span><span class="p">,</span> <span class="n">results</span><span class="p">):</span>
                    <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">res</span><span class="p">,</span> <span class="nb">Exception</span><span class="p">):</span>
                        <span class="k">continue</span>
                    <span class="n">content</span><span class="p">,</span> <span class="n">content_type</span> <span class="o">=</span> <span class="n">res</span>
                    
                    <span class="c1">#print(f"number of urls is /// {url} /// {content_type} /// {content}")
</span>                    <span class="k">if</span> <span class="n">content</span> <span class="ow">and</span> <span class="p">(</span><span class="n">content_type</span> <span class="ow">in</span> <span class="p">[</span><span class="s">"PDF"</span><span class="p">,</span> <span class="s">"HTML"</span><span class="p">]):</span>
                        <span class="n">extracted_content</span><span class="p">.</span><span class="n">append</span><span class="p">(</span>
                            <span class="sa">f</span><span class="s">"</span><span class="se">\n</span><span class="s">[Content from </span><span class="si">{</span><span class="n">url</span><span class="si">}</span><span class="s"> (</span><span class="si">{</span><span class="n">content_type</span><span class="si">}</span><span class="s">)]: </span><span class="si">{</span><span class="n">content</span><span class="p">[</span><span class="si">:</span><span class="n">MAX_CONTENT_LENGTH</span><span class="p">]</span><span class="si">}</span><span class="s">..."</span>
                        <span class="p">)</span>
                    <span class="k">elif</span> <span class="n">content</span> <span class="ow">and</span> <span class="p">(</span><span class="n">content_type</span> <span class="ow">in</span> <span class="p">[</span><span class="s">"Image"</span><span class="p">]):</span>
                        <span class="n">img_url</span> <span class="o">=</span> <span class="n">url</span>

                <span class="c1"># Combine everything
</span>                <span class="n">msg</span> <span class="o">=</span> <span class="n">msg</span><span class="p">[:</span><span class="n">MAX_DISCORD_LENGTH</span><span class="p">]</span>  <span class="c1"># Truncate user message if needed
</span>                <span class="n">msg</span> <span class="o">+=</span> <span class="s">''</span><span class="p">.</span><span class="n">join</span><span class="p">(</span><span class="n">extracted_content</span><span class="p">)</span>
                <span class="n">msg</span> <span class="o">+=</span> <span class="s">''</span><span class="p">.</span><span class="n">join</span><span class="p">(</span><span class="n">attached_text_list</span><span class="p">)</span>  <span class="c1"># &lt;-- CHANGED/ADDED
</span>
                <span class="k">print</span><span class="p">(</span><span class="s">"-User input------------------------------------------------------------------"</span><span class="p">)</span>
                <span class="k">print</span><span class="p">(</span><span class="sa">f</span><span class="s">"  Message content: '</span><span class="si">{</span><span class="n">msg</span><span class="si">}</span><span class="s">'"</span><span class="p">)</span>
                <span class="k">print</span><span class="p">(</span><span class="sa">f</span><span class="s">"  Image          : '</span><span class="si">{</span><span class="n">img_url</span><span class="si">}</span><span class="s">'"</span><span class="p">)</span>

                <span class="c1"># Build the user portion for conversation
</span>                <span class="n">discIn</span> <span class="o">=</span> <span class="p">[]</span>
                <span class="k">if</span> <span class="n">img_url</span><span class="p">:</span>
                    <span class="c1">#discIn.append({"role": "user", "content": f"{msg}\n(画像URL: {img_url})"})
</span>                    <span class="n">img_response</span> <span class="o">=</span> <span class="n">requests</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="n">img_url</span><span class="p">)</span>
                    <span class="n">base64_img</span>   <span class="o">=</span> <span class="n">base64</span><span class="p">.</span><span class="n">b64encode</span><span class="p">(</span><span class="n">img_response</span><span class="p">.</span><span class="n">content</span><span class="p">).</span><span class="n">decode</span><span class="p">(</span><span class="s">'utf-8'</span><span class="p">)</span>
                    <span class="n">data_url</span>     <span class="o">=</span> <span class="sa">f</span><span class="s">"data:image/png;base64,</span><span class="si">{</span><span class="n">base64_img</span><span class="si">}</span><span class="s">"</span>
                    <span class="n">discIn</span><span class="p">.</span><span class="n">append</span><span class="p">(</span>
                        <span class="p">{</span>
                            <span class="s">"role"</span><span class="p">:</span> <span class="s">"user"</span><span class="p">,</span>
                            <span class="s">"content"</span><span class="p">:</span> <span class="p">[</span>
                                <span class="p">{</span> <span class="s">"type"</span><span class="p">:</span> <span class="s">"text"</span><span class="p">,</span> <span class="s">"text"</span><span class="p">:</span> <span class="sa">f</span><span class="s">"msg"</span> <span class="p">},</span>
                                <span class="p">{</span>
                                    <span class="s">"type"</span><span class="p">:</span> <span class="s">"image_url"</span><span class="p">,</span>
                                    <span class="s">"image_url"</span><span class="p">:</span> <span class="p">{</span>
                                        <span class="s">"url"</span><span class="p">:</span> <span class="sa">f</span><span class="s">"</span><span class="si">{</span><span class="n">data_url</span><span class="si">}</span><span class="s">"</span><span class="p">,</span>
                                    <span class="p">},</span>
                                <span class="p">}</span>
                            <span class="p">],</span>
                        <span class="p">},</span>
                    <span class="p">)</span>
                <span class="k">else</span><span class="p">:</span>
                    <span class="k">if</span> <span class="n">msg</span><span class="p">:</span>
                        <span class="n">discIn</span><span class="p">.</span><span class="n">append</span><span class="p">({</span><span class="s">"role"</span><span class="p">:</span> <span class="s">"user"</span><span class="p">,</span> <span class="s">"content"</span><span class="p">:</span> <span class="n">msg</span><span class="p">})</span>

                <span class="n">conversation_history</span><span class="p">.</span><span class="n">extend</span><span class="p">(</span><span class="n">discIn</span><span class="p">)</span>

                <span class="c1"># Get AI response
</span>                <span class="n">response</span> <span class="o">=</span> <span class="k">await</span> <span class="n">ai_respond</span><span class="p">(</span><span class="n">discIn</span><span class="p">,</span> <span class="n">img_url</span><span class="p">)</span>

                <span class="c1"># Send response to Discord
</span>                <span class="k">await</span> <span class="n">send_long_message</span><span class="p">(</span><span class="n">message</span><span class="p">.</span><span class="n">channel</span><span class="p">,</span> <span class="n">response</span><span class="p">)</span>
                <span class="c1">#await message.channel.send(response)
</span>
                <span class="c1"># Add AI response to conversation history
</span>                <span class="n">conversation_history</span><span class="p">.</span><span class="n">append</span><span class="p">({</span><span class="s">"role"</span><span class="p">:</span> <span class="s">"assistant"</span><span class="p">,</span> <span class="s">"content"</span><span class="p">:</span> <span class="n">response</span><span class="p">})</span>
                <span class="k">print</span><span class="p">(</span><span class="s">"-Agent response--------------------------------------------------------------"</span><span class="p">)</span>
                <span class="k">print</span><span class="p">(</span><span class="sa">f</span><span class="s">"  Response content:'</span><span class="si">{</span><span class="n">response</span><span class="si">}</span><span class="s">'"</span><span class="p">)</span>


<span class="c1"># Initialize the client with the specified intents
</span><span class="n">d_client</span> <span class="o">=</span> <span class="n">MyClient</span><span class="p">(</span><span class="n">intents</span><span class="o">=</span><span class="n">intents</span><span class="p">)</span>
<span class="n">d_client</span><span class="p">.</span><span class="n">run</span><span class="p">(</span><span class="n">DISCORD_BOT_TOKEN</span><span class="p">)</span>
</code></pre></div></div>

<h2 id="run">run</h2>

<p>Run it in your local PC, no need for considerations on NAT etc..
The problem is, currently I do not have where is best fit for running always, as this discord.py requires</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>pip <span class="nb">install </span>discord.py requests python-dotenv asyncio openai beautifulsoup4 lxml PyPDF2 pycryptodome
python main.py
</code></pre></div></div>

<h2 id="detailed-explanation">Detailed explanation</h2>

<p>This project implements a Discord bot with capabilities to interact with OpenAI’s GPT models, fetch web content, and process file attachments. It supports concurrent operations, data extraction, and efficient handling of large Discord messages.</p>

<h3 id="features-overview">Features Overview</h3>

<ul>
  <li><strong>Environment Variable Handling:</strong> Loads required API keys and configuration from environment variables.</li>
  <li><strong>Message Processing:</strong> Handles messages, extracts URLs, fetches content, and generates responses.</li>
  <li><strong>OpenAI Integration:</strong> Processes messages with GPT and provides intelligent responses.</li>
  <li><strong>Web Content Fetching:</strong> Asynchronously fetches and processes content from URLs.</li>
  <li><strong>File Attachments Processing:</strong> Reads and extracts text from PDFs and HTML attachments.</li>
  <li><strong>Discord Bot Interaction:</strong> Handles commands, responses, and typing indicators.</li>
</ul>

<h3 id="functionality-breakdown">Functionality Breakdown</h3>

<h4 id="1-environment-variable-setup">1. Environment Variable Setup</h4>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">load_dotenv</span><span class="p">()</span>
<span class="n">DISCORD_BOT_TOKEN</span>    <span class="o">=</span> <span class="n">os</span><span class="p">.</span><span class="n">getenv</span><span class="p">(</span><span class="s">'DISCORD_BOT_TOKEN'</span><span class="p">)</span>
<span class="n">OPENAI_API_KEY</span>       <span class="o">=</span> <span class="n">os</span><span class="p">.</span><span class="n">getenv</span><span class="p">(</span><span class="s">'OPENAI_API_KEY'</span><span class="p">)</span>
<span class="n">BING_API_KEY</span>         <span class="o">=</span> <span class="n">os</span><span class="p">.</span><span class="n">getenv</span><span class="p">(</span><span class="s">'BING_API_KEY'</span><span class="p">)</span>
<span class="n">RESPOND_CHANNEL_NAME</span> <span class="o">=</span> <span class="n">os</span><span class="p">.</span><span class="n">getenv</span><span class="p">(</span><span class="s">'RESPOND_CHANNEL_NAME'</span><span class="p">)</span>
</code></pre></div></div>

<ul>
  <li>Loads API keys and settings using <code class="language-plaintext highlighter-rouge">dotenv</code> to keep sensitive information secure.</li>
</ul>

<h4 id="2-bot-intents-configuration">2. Bot Intents Configuration</h4>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">intents</span> <span class="o">=</span> <span class="n">discord</span><span class="p">.</span><span class="n">Intents</span><span class="p">.</span><span class="nb">all</span><span class="p">()</span>
<span class="n">intents</span><span class="p">.</span><span class="n">messages</span> <span class="o">=</span> <span class="bp">True</span>
<span class="n">intents</span><span class="p">.</span><span class="n">guilds</span> <span class="o">=</span> <span class="bp">True</span>
<span class="n">intents</span><span class="p">.</span><span class="n">message_content</span> <span class="o">=</span> <span class="bp">True</span>
</code></pre></div></div>

<ul>
  <li>Sets up the bot to receive message and guild-related events.</li>
</ul>

<h4 id="3-message-handling">3. Message Handling</h4>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">async</span> <span class="k">def</span> <span class="nf">on_message</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">message</span><span class="p">):</span>
    <span class="k">async</span> <span class="k">with</span> <span class="n">message</span><span class="p">.</span><span class="n">channel</span><span class="p">.</span><span class="n">typing</span><span class="p">():</span>
        <span class="n">msg</span> <span class="o">=</span> <span class="n">message</span><span class="p">.</span><span class="n">content</span>
        <span class="c1"># Process URLs and attachments
</span></code></pre></div></div>

<ul>
  <li>Processes incoming messages and checks for attachments and URLs.</li>
  <li>Displays the ‘typing’ indicator while processing.</li>
</ul>

<h4 id="4-url-content-fetching">4. URL Content Fetching</h4>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">async</span> <span class="k">def</span> <span class="nf">fetch_page_content_async</span><span class="p">(</span><span class="n">url</span><span class="p">):</span>
    <span class="n">response</span> <span class="o">=</span> <span class="k">await</span> <span class="n">asyncio</span><span class="p">.</span><span class="n">to_thread</span><span class="p">(</span><span class="n">requests</span><span class="p">.</span><span class="n">get</span><span class="p">,</span> <span class="n">url</span><span class="p">,</span> <span class="n">timeout</span><span class="o">=</span><span class="mi">10</span><span class="p">)</span>
    <span class="k">if</span> <span class="s">'text/html'</span> <span class="ow">in</span> <span class="n">response</span><span class="p">.</span><span class="n">headers</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="s">'Content-Type'</span><span class="p">,</span> <span class="s">''</span><span class="p">):</span>
        <span class="k">return</span> <span class="n">BeautifulSoup</span><span class="p">(</span><span class="n">response</span><span class="p">.</span><span class="n">content</span><span class="p">,</span> <span class="s">'lxml'</span><span class="p">).</span><span class="n">get_text</span><span class="p">(),</span> <span class="s">'HTML'</span>
</code></pre></div></div>

<ul>
  <li>Asynchronously fetches web content and extracts text from HTML or PDFs.</li>
  <li>Returns content and type (e.g., <code class="language-plaintext highlighter-rouge">HTML</code>, <code class="language-plaintext highlighter-rouge">PDF</code>).</li>
</ul>

<h4 id="5-handling-attachments">5. Handling Attachments</h4>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">async</span> <span class="k">def</span> <span class="nf">parse_discord_attachment</span><span class="p">(</span><span class="n">attachment</span><span class="p">):</span>
    <span class="n">file_bytes</span> <span class="o">=</span> <span class="k">await</span> <span class="n">attachment</span><span class="p">.</span><span class="n">read</span><span class="p">()</span>
    <span class="k">if</span> <span class="n">attachment</span><span class="p">.</span><span class="n">content_type</span><span class="p">.</span><span class="n">startswith</span><span class="p">(</span><span class="s">'application/pdf'</span><span class="p">):</span>
        <span class="k">return</span> <span class="n">extract_pdf_content</span><span class="p">(</span><span class="n">file_bytes</span><span class="p">),</span> <span class="s">"PDF"</span>
</code></pre></div></div>

<ul>
  <li>Reads and processes file attachments (PDFs and HTML) from Discord.</li>
</ul>

<h4 id="6-ai-response-generation">6. AI Response Generation</h4>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">async</span> <span class="k">def</span> <span class="nf">ai_respond</span><span class="p">(</span><span class="n">discIn</span><span class="p">,</span> <span class="n">img</span><span class="p">):</span>
    <span class="n">response</span> <span class="o">=</span> <span class="k">await</span> <span class="n">search_or_call_openai_async</span><span class="p">(</span><span class="n">discIn</span><span class="p">,</span> <span class="n">img</span><span class="p">)</span>
    <span class="k">return</span> <span class="n">response</span>
</code></pre></div></div>

<ul>
  <li>Determines whether to fetch external information or use internal knowledge.</li>
  <li>Calls OpenAI’s GPT model to generate responses.</li>
</ul>

<h4 id="7-search-and-summarization">7. Search and Summarization</h4>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">async</span> <span class="k">def</span> <span class="nf">summarize_results_async</span><span class="p">(</span><span class="n">search_results</span><span class="p">):</span>
    <span class="n">snippets</span> <span class="o">=</span> <span class="k">await</span> <span class="n">summarize_results_with_pages_async</span><span class="p">(</span><span class="n">search_results</span><span class="p">)</span>
    <span class="n">response</span> <span class="o">=</span> <span class="k">await</span> <span class="n">client</span><span class="p">.</span><span class="n">chat</span><span class="p">.</span><span class="n">completions</span><span class="p">.</span><span class="n">create</span><span class="p">(</span><span class="n">model</span><span class="o">=</span><span class="n">GPT_MODEL</span><span class="p">,</span> <span class="n">messages</span><span class="o">=</span><span class="n">messages</span><span class="p">)</span>
</code></pre></div></div>

<ul>
  <li>Searches reputable sources and summarizes the findings using GPT.</li>
</ul>

<h4 id="8-image-handling">8. Image Handling</h4>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">elif</span> <span class="n">content_type</span><span class="p">.</span><span class="n">startswith</span><span class="p">(</span><span class="s">'image/'</span><span class="p">):</span>
    <span class="n">base64_img</span> <span class="o">=</span> <span class="n">base64</span><span class="p">.</span><span class="n">b64encode</span><span class="p">(</span><span class="n">response</span><span class="p">.</span><span class="n">content</span><span class="p">).</span><span class="n">decode</span><span class="p">(</span><span class="s">'utf-8'</span><span class="p">)</span>
    <span class="k">return</span> <span class="sa">f</span><span class="s">"data:</span><span class="si">{</span><span class="n">content_type</span><span class="si">}</span><span class="s">;base64,</span><span class="si">{</span><span class="n">base64_img</span><span class="si">}</span><span class="s">"</span><span class="p">,</span> <span class="s">"Image"</span>
</code></pre></div></div>

<ul>
  <li>Converts image URLs to base64-encoded data URLs for embedding in AI responses.</li>
</ul>

<h4 id="9-sending-responses">9. Sending Responses</h4>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">async</span> <span class="k">def</span> <span class="nf">send_long_message</span><span class="p">(</span><span class="n">channel</span><span class="p">,</span> <span class="n">content</span><span class="p">):</span>
    <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">content</span><span class="p">),</span> <span class="n">MAX_DISCORD_REPLY_LENGTH</span><span class="p">):</span>
        <span class="k">await</span> <span class="n">channel</span><span class="p">.</span><span class="n">send</span><span class="p">(</span><span class="n">content</span><span class="p">[</span><span class="n">i</span><span class="p">:</span> <span class="n">i</span> <span class="o">+</span> <span class="n">MAX_DISCORD_REPLY_LENGTH</span><span class="p">])</span>
</code></pre></div></div>

<ul>
  <li>Splits and sends messages in chunks to adhere to Discord’s character limits.</li>
</ul>

<h4 id="10-conversation-history">10. Conversation History</h4>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">conversation_history</span> <span class="o">=</span> <span class="n">deque</span><span class="p">(</span><span class="n">maxlen</span><span class="o">=</span><span class="n">HISTORY_LENGTH</span><span class="p">)</span>
</code></pre></div></div>

<ul>
  <li>Maintains a rolling history of previous messages to provide context in responses.</li>
</ul>

<h4 id="11-running-the-bot">11. Running the Bot</h4>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">d_client</span> <span class="o">=</span> <span class="n">MyClient</span><span class="p">(</span><span class="n">intents</span><span class="o">=</span><span class="n">intents</span><span class="p">)</span>
<span class="n">d_client</span><span class="p">.</span><span class="n">run</span><span class="p">(</span><span class="n">DISCORD_BOT_TOKEN</span><span class="p">)</span>
</code></pre></div></div>

<ul>
  <li>Initializes the bot and starts running with specified intents.</li>
</ul>

<h3 id="summary">Summary</h3>

<p>This bot integrates AI and web content features to provide intelligent responses, retrieve useful information from the web, and process attachments efficiently. By leveraging async operations, it ensures smooth performance and a responsive user experience.</p>]]></content><author><name>Tomohiko Araki</name></author><summary type="html"><![CDATA[I have made server for my family, and there shoud be some cute companions like kitten, so I have prepared Bot with a unique identity and memoty of recent conversation.]]></summary></entry><entry><title type="html">Upload files from GCP to Azure without SAS/AccessKey</title><link href="https://metaproxy.co/AzureGCPFederation/" rel="alternate" type="text/html" title="Upload files from GCP to Azure without SAS/AccessKey" /><published>2024-02-24T00:00:00+00:00</published><updated>2024-02-24T00:00:00+00:00</updated><id>https://metaproxy.co/AzureGCPFederation</id><content type="html" xml:base="https://metaproxy.co/AzureGCPFederation/"><![CDATA[<p><img src="https://github.com/rtree/pages/assets/1018794/5891e09b-69e8-47e2-af3e-778883314b52" alt="image" /></p>

<p>Credential is like plutonium.</p>

<p>One concerns recently on me is that User account for human-kinds are secured by MFA, but what about workload identities / Service accounts?
They are used within scripts stored securely, but once they are exfiltrated, they can be utilized from anywhere basically, this is the issue.</p>

<p>While looking around the cloud trends, recent years, we can use:</p>
<ul>
  <li>Service accounts without keys within GCP</li>
  <li>User assigned managed identities within Azure</li>
</ul>

<p>How it would be nice if we can do above in cross-cloud scenario! and there is the way for it.
I have tried cross cloud for uploading files from GCP to Azure Data Lake Gen2, and access them from PowerBI Desktop.</p>

<h2 id="how-it-is-made-of">How it is made of</h2>

<p>Just a few steps:</p>
<ul>
  <li>Prepare accounts in each cloud: user-assigned managed-identities in Azure, service account in GCP</li>
  <li>Connect above 2 by setting federated identitiy into user-assigned managed-identities in Azure</li>
  <li>Get token from GCP, pass it to Azure for returning back of token of Azure</li>
  <li>Upload files to Azure Data Lake Gen2 using the returned token of Azure</li>
</ul>

<p>The steps above is well documented in following:</p>
<ul>
  <li><a href="https://learn.microsoft.com/en-us/entra/workload-id/workload-identity-federation">https://learn.microsoft.com/en-us/entra/workload-id/workload-identity-federation</a></li>
  <li><a href="https://blog.identitydigest.com/azuread-federate-gcp/">https://blog.identitydigest.com/azuread-federate-gcp/</a></li>
</ul>

<p>Well documented, but it was a little bit hard to grab overview and small how-to-s for actual implementation.
That’s why I have introduced what I did in this article.</p>

<h2 id="what-i-did">What I did</h2>

<h3 id="procedure">Procedure</h3>

<h4 id="configure-in-gcp">[Configure in GCP]</h4>

<ul>
  <li>1) Create service account</li>
  <li>2) Assign it as Service Account of CloudFunctions</li>
  <li>3) Refer cloudFunctions part</li>
</ul>

<h4 id="configure-in-azure">[Configure in Azure]</h4>

<ul>
  <li>1) Create user-assigned managed identity</li>
  <li>2) Set federated credentials to it
    <ul>
      <li>Federated credential scenario: OpenID Connect</li>
      <li>Issuer URL: https://accounts.google.com</li>
      <li>Subject identifier: Unique ID of the GCP service account</li>
      <li>Name: (Assign as you like)</li>
      <li>Audience: api://AzureADTokenExchange</li>
    </ul>
  </li>
  <li>3) Make storageAccount with hierarcy enabled (to be accessed as Azure Data Lake Gen2)</li>
  <li>4) Assign Storage Data Contributer of storageAccount to user-assigned managed identity</li>
</ul>

<h4 id="place-scripts-into-cloudfunctions">[Place scripts into CloudFunctions]</h4>

<p>refer following</p>

<h3 id="mainpy">main.py</h3>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">functions_framework</span>
<span class="kn">from</span> <span class="nn">google.cloud</span> <span class="kn">import</span> <span class="n">storage</span>
<span class="kn">from</span> <span class="nn">google.cloud</span> <span class="kn">import</span> <span class="n">firestore</span>
<span class="kn">import</span> <span class="nn">json</span>
<span class="kn">import</span> <span class="nn">os</span>
<span class="kn">from</span> <span class="nn">google.cloud.firestore</span> <span class="kn">import</span> <span class="n">SERVER_TIMESTAMP</span>
<span class="kn">from</span> <span class="nn">datetime</span> <span class="kn">import</span> <span class="n">datetime</span>
<span class="kn">import</span> <span class="nn">requests</span>
<span class="kn">from</span> <span class="nn">azure.storage.filedatalake</span> <span class="kn">import</span> <span class="n">DataLakeServiceClient</span>
<span class="kn">from</span> <span class="nn">azure.identity</span> <span class="kn">import</span> <span class="n">ClientAssertionCredential</span>
<span class="kn">import</span> <span class="nn">logging</span>
<span class="n">BUCKET_NAME</span> <span class="o">=</span> <span class="n">os</span><span class="p">.</span><span class="n">getenv</span><span class="p">(</span><span class="s">"BUCKET_NAME"</span><span class="p">)</span>
<span class="n">FOLDER_NAME</span> <span class="o">=</span> <span class="n">os</span><span class="p">.</span><span class="n">getenv</span><span class="p">(</span><span class="s">"FOLDER_NAME"</span><span class="p">)</span>
<span class="n">AZURE_TENANT_ID</span>      <span class="o">=</span> <span class="n">os</span><span class="p">.</span><span class="n">getenv</span><span class="p">(</span><span class="s">"AZURE_TENANT_ID"</span><span class="p">)</span>
<span class="n">AZURE_CLIENT_ID</span>      <span class="o">=</span> <span class="n">os</span><span class="p">.</span><span class="n">getenv</span><span class="p">(</span><span class="s">"AZURE_CLIENT_ID"</span><span class="p">)</span>
<span class="n">AZURE_DATALAKE_URL</span>   <span class="o">=</span> <span class="n">os</span><span class="p">.</span><span class="n">getenv</span><span class="p">(</span><span class="s">"AZURE_DATALAKE_URL"</span><span class="p">)</span>
<span class="n">AZURE_CONTAINER_NAME</span> <span class="o">=</span> <span class="n">os</span><span class="p">.</span><span class="n">getenv</span><span class="p">(</span><span class="s">"AZURE_CONTAINER_NAME"</span><span class="p">)</span>
<span class="o">@</span><span class="n">functions_framework</span><span class="p">.</span><span class="n">http</span>
<span class="k">def</span> <span class="nf">hello_http</span><span class="p">(</span><span class="n">request</span><span class="p">):</span>
    <span class="s">"""HTTP Cloud Function.
    Args:
        request (flask.Request): The request object.
        &lt;https://flask.palletsprojects.com/en/1.1.x/api/#incoming-request-data&gt;
    Returns:
        The response text, or any set of values that can be turned into a
        Response object using `make_response`
        &lt;https://flask.palletsprojects.com/en/1.1.x/api/#flask.make_response&gt;.
    """</span>
    <span class="n">request_json</span> <span class="o">=</span> <span class="n">request</span><span class="p">.</span><span class="n">get_json</span><span class="p">(</span><span class="n">silent</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
    <span class="n">request_args</span> <span class="o">=</span> <span class="n">request</span><span class="p">.</span><span class="n">args</span>
    <span class="k">if</span> <span class="n">request_json</span> <span class="ow">and</span> <span class="s">'name'</span> <span class="ow">in</span> <span class="n">request_json</span><span class="p">:</span>
        <span class="n">name</span> <span class="o">=</span> <span class="n">request_json</span><span class="p">[</span><span class="s">'name'</span><span class="p">]</span>
    <span class="k">elif</span> <span class="n">request_args</span> <span class="ow">and</span> <span class="s">'name'</span> <span class="ow">in</span> <span class="n">request_args</span><span class="p">:</span>
        <span class="n">name</span> <span class="o">=</span> <span class="n">request_args</span><span class="p">[</span><span class="s">'name'</span><span class="p">]</span>
    <span class="k">else</span><span class="p">:</span>
        <span class="n">name</span> <span class="o">=</span> <span class="s">'World'</span>
    <span class="n">save_to_bucket</span><span class="p">(</span><span class="n">BUCKET_NAME</span><span class="p">,</span> <span class="n">FOLDER_NAME</span><span class="p">,</span> <span class="s">"_log_launch"</span><span class="p">)</span>
    <span class="k">return</span> <span class="s">'Hello {}!'</span><span class="p">.</span><span class="nb">format</span><span class="p">(</span><span class="n">name</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">mask_field</span><span class="p">(</span><span class="n">field_value</span><span class="p">):</span>
    <span class="c1"># TODO: Add your masking logic here
</span>    <span class="c1"># This is just an example that replaces the field value with asterisks
</span>    <span class="n">masked</span> <span class="o">=</span> <span class="n">field_value</span>
    <span class="k">return</span> <span class="n">masked</span>
<span class="k">def</span> <span class="nf">to_serializable</span><span class="p">(</span><span class="n">val</span><span class="p">):</span>
    <span class="s">"""
    Convert Firestore data types to JSON serializable objects
    """</span>
    <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">val</span><span class="p">,</span> <span class="n">datetime</span><span class="p">):</span>
        <span class="k">return</span> <span class="n">val</span><span class="p">.</span><span class="n">isoformat</span><span class="p">()</span>
    <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">val</span><span class="p">,</span> <span class="n">SERVER_TIMESTAMP</span><span class="p">):</span>
        <span class="k">return</span> <span class="bp">None</span>
    <span class="k">else</span><span class="p">:</span>
        <span class="k">return</span> <span class="n">val</span>
<span class="k">def</span> <span class="nf">dump_to_blob</span><span class="p">(</span><span class="n">filename</span><span class="p">,</span> <span class="n">str_to_dump</span><span class="p">):</span>
    <span class="n">db</span> <span class="o">=</span> <span class="n">firestore</span><span class="p">.</span><span class="n">Client</span><span class="p">()</span>
    <span class="n">storage_client</span> <span class="o">=</span> <span class="n">storage</span><span class="p">.</span><span class="n">Client</span><span class="p">()</span>
    <span class="n">bucket</span> <span class="o">=</span> <span class="n">storage_client</span><span class="p">.</span><span class="n">bucket</span><span class="p">(</span><span class="n">BUCKET_NAME</span><span class="p">)</span>
    <span class="n">blob</span> <span class="o">=</span> <span class="n">bucket</span><span class="p">.</span><span class="n">blob</span><span class="p">(</span><span class="n">filename</span> <span class="o">+</span> <span class="s">"dump.txt"</span><span class="p">)</span>
    <span class="n">blob</span><span class="p">.</span><span class="n">upload_from_string</span><span class="p">(</span><span class="n">str_to_dump</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">save_to_adls_sdk</span><span class="p">(</span><span class="n">credential</span><span class="p">):</span>
    <span class="n">service_client</span> <span class="o">=</span> <span class="n">DataLakeServiceClient</span><span class="p">(</span><span class="n">account_url</span><span class="o">=</span><span class="n">AZURE_DATALAKE_URL</span><span class="p">,</span>
                                           <span class="n">credential</span> <span class="o">=</span><span class="n">credential</span><span class="p">)</span>
    <span class="n">file_system_client</span> <span class="o">=</span> <span class="n">service_client</span><span class="p">.</span><span class="n">get_file_system_client</span><span class="p">(</span><span class="n">file_system</span><span class="o">=</span><span class="n">AZURE_CONTAINER_NAME</span><span class="p">)</span>
    <span class="n">file_name</span> <span class="o">=</span> <span class="s">"example.txt"</span>
    <span class="n">file_contents</span> <span class="o">=</span> <span class="s">"Hello, Azure AD and GCP Federation!"</span>
    <span class="n">file_client</span> <span class="o">=</span> <span class="n">file_system_client</span><span class="p">.</span><span class="n">get_file_client</span><span class="p">(</span><span class="n">file_name</span><span class="p">)</span>
    <span class="n">file_client</span><span class="p">.</span><span class="n">upload_data</span><span class="p">(</span><span class="n">file_contents</span><span class="p">,</span> <span class="n">overwrite</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
    <span class="k">print</span><span class="p">(</span><span class="sa">f</span><span class="s">"Uploaded </span><span class="si">{</span><span class="n">file_name</span><span class="si">}</span><span class="s"> to Azure Data Lake Storage Gen2"</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">get_azure_ad_token_sdk</span><span class="p">():</span>
    <span class="n">credential</span> <span class="o">=</span> <span class="n">ClientAssertionCredential</span><span class="p">(</span>
        <span class="n">AZURE_TENANT_ID</span><span class="p">,</span>
        <span class="n">AZURE_CLIENT_ID</span><span class="p">,</span>
        <span class="n">func</span><span class="o">=</span><span class="n">get_google_id_token</span><span class="p">,</span>
    <span class="p">)</span>
    <span class="k">return</span> <span class="n">credential</span>
<span class="k">def</span> <span class="nf">get_google_id_token</span><span class="p">():</span>
    <span class="n">metadata_url</span> <span class="o">=</span> <span class="s">"http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/identity"</span>
    <span class="n">params</span> <span class="o">=</span> <span class="p">{</span>
        <span class="s">"audience"</span><span class="p">:</span> <span class="s">"api://AzureADTokenExchange"</span>
    <span class="p">}</span>
    <span class="n">headers</span> <span class="o">=</span> <span class="p">{</span>
        <span class="s">"Metadata-Flavor"</span><span class="p">:</span> <span class="s">"Google"</span>
    <span class="p">}</span>
    <span class="n">response</span> <span class="o">=</span> <span class="n">requests</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="n">metadata_url</span><span class="p">,</span> <span class="n">params</span><span class="o">=</span><span class="n">params</span><span class="p">,</span> <span class="n">headers</span><span class="o">=</span><span class="n">headers</span><span class="p">)</span>
    <span class="k">if</span> <span class="n">response</span><span class="p">.</span><span class="n">status_code</span> <span class="o">==</span> <span class="mi">200</span><span class="p">:</span>
      <span class="k">return</span> <span class="n">response</span><span class="p">.</span><span class="n">text</span>
    <span class="k">else</span><span class="p">:</span>
        <span class="k">raise</span> <span class="nb">Exception</span><span class="p">(</span><span class="s">"Failed to obtain Google ID token"</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">save_to_bucket</span><span class="p">(</span><span class="n">bucket_name</span><span class="p">,</span> <span class="n">folder_name</span><span class="p">,</span> <span class="n">collection_name</span><span class="p">):</span>
    <span class="n">acred</span> <span class="o">=</span> <span class="n">get_azure_ad_token_sdk</span><span class="p">()</span>
    <span class="n">logging</span><span class="p">.</span><span class="n">basicConfig</span><span class="p">(</span><span class="n">level</span><span class="o">=</span><span class="n">logging</span><span class="p">.</span><span class="n">DEBUG</span><span class="p">)</span>
    <span class="n">save_to_adls_sdk</span><span class="p">(</span><span class="n">acred</span><span class="p">)</span>
</code></pre></div></div>

<h3 id="requirementstxt">requirements.txt</h3>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">functions</span><span class="o">-</span><span class="n">framework</span><span class="o">==</span><span class="mf">3.</span><span class="o">*</span>
<span class="n">google</span><span class="o">-</span><span class="n">cloud</span><span class="o">-</span><span class="n">firestore</span>
<span class="n">google</span><span class="o">-</span><span class="n">cloud</span><span class="o">-</span><span class="n">storage</span>
<span class="n">requests</span>
<span class="n">azure</span><span class="o">-</span><span class="n">storage</span><span class="o">-</span><span class="nb">file</span><span class="o">-</span><span class="n">datalake</span>
<span class="n">azure</span><span class="o">-</span><span class="n">identity</span>
</code></pre></div></div>

<h3 id="variables">variables</h3>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>AZURE_TENANT_ID      Entra-&gt;Identity-&gt;Overview-&gt;Tenant ID						
AZURE_CLIENT_ID      Managed Identities-&gt; <span class="o">(</span>the identity created<span class="o">)</span>-&gt;overview-&gt;Client ID						
AZURE_DATALAKE_URL      https://&lt;your-storage-account&gt;.dfs.core.windows.net/						
AZURE_CONTAINER_NAME      Your Container Name
</code></pre></div></div>

<h2 id="conclusion">Conclusion</h2>

<p>Enjoy! and if you have questions, please DM on <a href="https://twitter.com/rtree">https://twitter.com/rtree</a>.</p>]]></content><author><name>Tomohiko Araki</name></author><summary type="html"><![CDATA[]]></summary></entry><entry><title type="html">DMARC Report prosessing</title><link href="https://metaproxy.co/dmarc/" rel="alternate" type="text/html" title="DMARC Report prosessing" /><published>2024-02-19T00:00:00+00:00</published><updated>2024-02-19T00:00:00+00:00</updated><id>https://metaproxy.co/dmarc</id><content type="html" xml:base="https://metaproxy.co/dmarc/"><![CDATA[<p>I just would like to have DMARC report into 1 folder, and make them packed into CSV, as popular DMARC processing SaaS is a little bit costy.</p>

<h2 id="code-it-into-dmarcpy">code it into dmarc.py</h2>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">os</span>
<span class="kn">import</span> <span class="nn">gzip</span>
<span class="kn">import</span> <span class="nn">zipfile</span>
<span class="kn">from</span> <span class="nn">lxml</span> <span class="kn">import</span> <span class="n">etree</span>
<span class="kn">import</span> <span class="nn">csv</span>
<span class="kn">import</span> <span class="nn">datetime</span>

<span class="k">def</span> <span class="nf">extract_info_from_filename</span><span class="p">(</span><span class="n">filename</span><span class="p">):</span>
    <span class="s">"""
    Extracts start and end datetimes from a given filename.
    Args:
    - filename (str): The filename containing Unix timestamps for start and end times.
    - filename expecting..: 'google.com!domainname!1707004800!1707091199.xml'
    Returns:
    - tuple: A tuple containing start and end datetime objects.
    """</span>
    <span class="c1"># Assuming filenames are like report_starttime_endtime.xml
</span>    <span class="c1"># Extract timestamps from filename
</span>    <span class="n">parts</span> <span class="o">=</span> <span class="n">filename</span><span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">".gz"</span><span class="p">,</span> <span class="s">""</span><span class="p">).</span><span class="n">replace</span><span class="p">(</span><span class="s">".xml"</span><span class="p">,</span> <span class="s">""</span><span class="p">).</span><span class="n">split</span><span class="p">(</span><span class="s">"!"</span><span class="p">)</span>
    <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">parts</span><span class="p">)</span> <span class="o">&gt;=</span> <span class="mi">4</span><span class="p">:</span>
        <span class="n">to_domain</span><span class="p">,</span> <span class="n">from_domain</span><span class="p">,</span> <span class="n">start_timestamp</span><span class="p">,</span> <span class="n">end_timestamp</span><span class="p">,</span> <span class="o">*</span><span class="n">other_abysss</span> <span class="o">=</span> <span class="n">parts</span>
        <span class="k">try</span><span class="p">:</span>
            <span class="n">start_datetime</span> <span class="o">=</span> <span class="n">datetime</span><span class="p">.</span><span class="n">datetime</span><span class="p">.</span><span class="n">fromtimestamp</span><span class="p">(</span><span class="nb">int</span><span class="p">(</span><span class="n">start_timestamp</span><span class="p">))</span>
            <span class="n">end_datetime</span>   <span class="o">=</span> <span class="n">datetime</span><span class="p">.</span><span class="n">datetime</span><span class="p">.</span><span class="n">fromtimestamp</span><span class="p">(</span><span class="nb">int</span><span class="p">(</span><span class="n">end_timestamp</span><span class="p">))</span>
        <span class="k">except</span> <span class="nb">ValueError</span><span class="p">:</span>
            <span class="n">start_datetime</span> <span class="o">=</span> <span class="n">datetime</span><span class="p">.</span><span class="n">datetime</span><span class="p">.</span><span class="n">fromtimestamp</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
            <span class="n">end_datetime</span>   <span class="o">=</span> <span class="n">datetime</span><span class="p">.</span><span class="n">datetime</span><span class="p">.</span><span class="n">fromtimestamp</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span>
    <span class="k">else</span><span class="p">:</span>
        <span class="n">to_domain</span>          <span class="o">=</span> <span class="s">'to.example.com'</span>
        <span class="n">from_domain</span>        <span class="o">=</span> <span class="s">'from.example.com'</span>
        <span class="n">start_datetime</span>     <span class="o">=</span> <span class="n">datetime</span><span class="p">.</span><span class="n">datetime</span><span class="p">.</span><span class="n">fromtimestamp</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
        <span class="n">end_datetime</span>       <span class="o">=</span> <span class="n">datetime</span><span class="p">.</span><span class="n">datetime</span><span class="p">.</span><span class="n">fromtimestamp</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span>
    <span class="k">return</span> <span class="n">to_domain</span><span class="p">,</span> <span class="n">from_domain</span><span class="p">,</span> <span class="n">start_datetime</span><span class="p">,</span> <span class="n">end_datetime</span>

<span class="k">def</span> <span class="nf">extract_and_parse_files</span><span class="p">(</span><span class="n">directory</span><span class="p">,</span> <span class="n">output_csv</span><span class="p">):</span>
    <span class="c1"># Open or create the CSV file for output
</span>    <span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="n">output_csv</span><span class="p">,</span> <span class="s">'w'</span><span class="p">,</span> <span class="n">newline</span><span class="o">=</span><span class="s">''</span><span class="p">,</span> <span class="n">encoding</span><span class="o">=</span><span class="s">'utf-8'</span><span class="p">)</span> <span class="k">as</span> <span class="n">csvfile</span><span class="p">:</span>
        <span class="n">csvwriter</span> <span class="o">=</span> <span class="n">csv</span><span class="p">.</span><span class="n">writer</span><span class="p">(</span><span class="n">csvfile</span><span class="p">)</span>
        <span class="c1"># Write the CSV header
</span>        <span class="n">csvwriter</span><span class="p">.</span><span class="n">writerow</span><span class="p">([</span><span class="s">'Filename'</span><span class="p">,</span> <span class="s">'src'</span><span class="p">,</span> <span class="s">'count'</span><span class="p">,</span> <span class="s">'Header-From Domain'</span><span class="p">,</span> <span class="s">'SPF Domnain'</span><span class="p">,</span> <span class="s">'SPF check'</span><span class="p">,</span> <span class="s">'SPF alignment'</span><span class="p">,</span> <span class="s">'DKIM Domain'</span><span class="p">,</span> <span class="s">'DKIM check'</span><span class="p">,</span> <span class="s">'DKIM alignment'</span> <span class="p">,</span> <span class="s">'Detail'</span><span class="p">,</span> <span class="s">'DMARC disposition'</span><span class="p">,</span> <span class="s">'To'</span><span class="p">,</span> <span class="s">'From'</span><span class="p">,</span> <span class="s">'Start'</span><span class="p">,</span> <span class="s">'End'</span><span class="p">])</span>
        <span class="c1"># Process each file in the directory
</span>        <span class="k">for</span> <span class="n">root</span><span class="p">,</span> <span class="n">dirs</span><span class="p">,</span> <span class="n">files</span> <span class="ow">in</span> <span class="n">os</span><span class="p">.</span><span class="n">walk</span><span class="p">(</span><span class="n">directory</span><span class="p">):</span>
            <span class="k">for</span> <span class="nb">file</span> <span class="ow">in</span> <span class="n">files</span><span class="p">:</span>
                <span class="n">file_path</span> <span class="o">=</span> <span class="n">os</span><span class="p">.</span><span class="n">path</span><span class="p">.</span><span class="n">join</span><span class="p">(</span><span class="n">root</span><span class="p">,</span> <span class="nb">file</span><span class="p">)</span>
                <span class="c1"># Check if the file is a .gz or .zip file and extract XML content
</span>                <span class="k">if</span> <span class="nb">file</span><span class="p">.</span><span class="n">endswith</span><span class="p">(</span><span class="s">'.gz'</span><span class="p">):</span>
                    <span class="k">with</span> <span class="n">gzip</span><span class="p">.</span><span class="nb">open</span><span class="p">(</span><span class="n">file_path</span><span class="p">,</span> <span class="s">'rb'</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
                        <span class="n">xml_content</span> <span class="o">=</span> <span class="n">f</span><span class="p">.</span><span class="n">read</span><span class="p">()</span>
                        <span class="n">parse_xml_content</span><span class="p">(</span><span class="n">xml_content</span><span class="p">,</span> <span class="nb">file</span><span class="p">,</span> <span class="n">csvwriter</span><span class="p">)</span>
                <span class="k">elif</span> <span class="nb">file</span><span class="p">.</span><span class="n">endswith</span><span class="p">(</span><span class="s">'.zip'</span><span class="p">):</span>
                    <span class="k">with</span> <span class="n">zipfile</span><span class="p">.</span><span class="n">ZipFile</span><span class="p">(</span><span class="n">file_path</span><span class="p">,</span> <span class="s">'r'</span><span class="p">)</span> <span class="k">as</span> <span class="n">zip_ref</span><span class="p">:</span>
                        <span class="k">for</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">zip_ref</span><span class="p">.</span><span class="n">namelist</span><span class="p">():</span>
                            <span class="k">with</span> <span class="n">zip_ref</span><span class="p">.</span><span class="nb">open</span><span class="p">(</span><span class="n">name</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
                                <span class="n">xml_content</span> <span class="o">=</span> <span class="n">f</span><span class="p">.</span><span class="n">read</span><span class="p">()</span>
                                <span class="n">parse_xml_content</span><span class="p">(</span><span class="n">xml_content</span><span class="p">,</span> <span class="n">name</span><span class="p">,</span> <span class="n">csvwriter</span><span class="p">)</span>

<span class="k">def</span> <span class="nf">parse_xml_content</span><span class="p">(</span><span class="n">xml_content</span><span class="p">,</span> <span class="n">filename</span><span class="p">,</span> <span class="n">csvwriter</span><span class="p">):</span>
    <span class="c1"># Parse the XML content
</span>    <span class="n">root</span> <span class="o">=</span> <span class="n">etree</span><span class="p">.</span><span class="n">fromstring</span><span class="p">(</span><span class="n">xml_content</span><span class="p">)</span>
    <span class="c1"># Iterate through the records in the XML
</span>    <span class="k">for</span> <span class="n">record</span> <span class="ow">in</span> <span class="n">root</span><span class="p">.</span><span class="n">findall</span><span class="p">(</span><span class="s">'.//record'</span><span class="p">):</span>
        <span class="n">row</span>              <span class="o">=</span> <span class="n">record</span><span class="p">.</span><span class="n">find</span><span class="p">(</span><span class="s">'.//row'</span><span class="p">)</span>
        <span class="n">source_ip</span>           <span class="o">=</span> <span class="n">row</span><span class="p">.</span><span class="n">find</span><span class="p">(</span><span class="s">'.//source_ip'</span><span class="p">).</span><span class="n">text</span>
        <span class="n">count</span>               <span class="o">=</span> <span class="n">row</span><span class="p">.</span><span class="n">find</span><span class="p">(</span><span class="s">'.//count'</span><span class="p">).</span><span class="n">text</span>
        <span class="n">policy_evaluated</span>    <span class="o">=</span> <span class="n">row</span><span class="p">.</span><span class="n">find</span><span class="p">(</span><span class="s">'.//policy_evaluated'</span><span class="p">)</span>
        <span class="n">identifiers</span>      <span class="o">=</span> <span class="n">record</span><span class="p">.</span><span class="n">find</span><span class="p">(</span><span class="s">'.//identifiers'</span><span class="p">)</span>
        <span class="n">header_from</span>         <span class="o">=</span> <span class="n">identifiers</span><span class="p">.</span><span class="n">find</span><span class="p">(</span><span class="s">'.//header_from'</span><span class="p">).</span><span class="n">text</span> <span class="k">if</span> <span class="n">identifiers</span> <span class="ow">is</span> <span class="ow">not</span> <span class="bp">None</span> <span class="k">else</span> <span class="s">'Unknown'</span>
        <span class="c1"># DMARC evaluation
</span>        <span class="n">dmarc_fail</span> <span class="o">=</span> <span class="n">policy_evaluated</span><span class="p">.</span><span class="n">find</span><span class="p">(</span><span class="s">'.//disposition'</span><span class="p">).</span><span class="n">text</span>
        <span class="c1"># SPF evaluation
</span>        <span class="n">spf_result</span>       <span class="o">=</span> <span class="n">record</span><span class="p">.</span><span class="n">find</span><span class="p">(</span><span class="s">'.//auth_results//spf//result'</span><span class="p">).</span><span class="n">text</span> <span class="k">if</span> <span class="n">record</span><span class="p">.</span><span class="n">find</span><span class="p">(</span><span class="s">'.//auth_results//spf//result'</span><span class="p">)</span> <span class="ow">is</span> <span class="ow">not</span> <span class="bp">None</span> <span class="k">else</span> <span class="s">'n/a'</span>
        <span class="n">spf_fail</span>         <span class="o">=</span> <span class="n">spf_result</span>
        <span class="n">spf_domain</span>       <span class="o">=</span> <span class="n">record</span><span class="p">.</span><span class="n">find</span><span class="p">(</span><span class="s">'.//auth_results//spf//domain'</span><span class="p">).</span><span class="n">text</span> <span class="k">if</span> <span class="n">record</span><span class="p">.</span><span class="n">find</span><span class="p">(</span><span class="s">'.//auth_results//spf//domain'</span><span class="p">)</span> <span class="ow">is</span> <span class="ow">not</span> <span class="bp">None</span> <span class="k">else</span> <span class="s">'n/a'</span>
        <span class="n">spf_aligned</span>      <span class="o">=</span> <span class="s">'pass'</span> <span class="k">if</span> <span class="n">spf_domain</span> <span class="o">==</span> <span class="n">header_from</span> <span class="k">else</span> <span class="s">'fail'</span>

        <span class="c1"># # DKIM evaluation
</span>        <span class="c1"># dkim_result      = record.find('.//auth_results//dkim//result').text if record.find('.//auth_results//dkim//result') is not None else 'n/a'
</span>        <span class="c1"># dkim_fail        = dkim_result
</span>        <span class="c1"># dkim_domain      = record.find('.//auth_results//dkim//domain').text if record.find('.//auth_results//dkim//domain') is not None else 'n/a'
</span>        <span class="c1"># dkim_aligned     = 'pass' if dkim_domain == header_from else 'fail'
</span>
        <span class="n">dkim_detail</span>  <span class="o">=</span> <span class="s">''</span>
        <span class="n">dkim_fail</span>    <span class="o">=</span> <span class="s">''</span>
        <span class="n">dkim_domain</span>  <span class="o">=</span> <span class="s">''</span>
        <span class="n">dkim_aligned</span> <span class="o">=</span> <span class="s">''</span>
        <span class="k">for</span> <span class="n">dkim</span> <span class="ow">in</span> <span class="n">record</span><span class="p">.</span><span class="n">findall</span><span class="p">(</span><span class="s">'.//auth_results//dkim'</span><span class="p">):</span>
            <span class="n">d_result</span> <span class="o">=</span> <span class="n">dkim</span><span class="p">.</span><span class="n">find</span><span class="p">(</span><span class="s">'.//result'</span><span class="p">).</span><span class="n">text</span> <span class="k">if</span> <span class="n">dkim</span><span class="p">.</span><span class="n">find</span><span class="p">(</span><span class="s">'.//result'</span><span class="p">)</span> <span class="ow">is</span> <span class="ow">not</span> <span class="bp">None</span> <span class="k">else</span> <span class="s">'none'</span>
            <span class="n">d_domain</span> <span class="o">=</span> <span class="n">dkim</span><span class="p">.</span><span class="n">find</span><span class="p">(</span><span class="s">'.//domain'</span><span class="p">).</span><span class="n">text</span> <span class="k">if</span> <span class="n">dkim</span><span class="p">.</span><span class="n">find</span><span class="p">(</span><span class="s">'.//domain'</span><span class="p">)</span> <span class="ow">is</span> <span class="ow">not</span> <span class="bp">None</span> <span class="k">else</span> <span class="s">'none'</span>
            <span class="n">d_aligned</span><span class="o">=</span> <span class="s">'pass'</span> <span class="k">if</span> <span class="n">d_domain</span> <span class="o">==</span> <span class="n">header_from</span> <span class="k">else</span> <span class="s">'fail'</span>
            <span class="n">dkim_detail</span> <span class="o">+=</span>  <span class="n">d_result</span> <span class="o">+</span> <span class="s">';'</span> <span class="o">+</span> <span class="n">d_domain</span> <span class="o">+</span> <span class="s">';'</span> <span class="o">+</span> <span class="n">d_aligned</span> <span class="o">+</span> <span class="s">';  '</span>
            <span class="k">if</span> <span class="n">dkim_fail</span> <span class="o">==</span> <span class="s">''</span> <span class="ow">and</span> <span class="n">dkim_domain</span> <span class="o">==</span> <span class="s">''</span> <span class="ow">and</span> <span class="n">dkim_aligned</span> <span class="o">==</span> <span class="s">''</span><span class="p">:</span>
                <span class="n">dkim_fail</span>    <span class="o">=</span> <span class="n">d_result</span>
                <span class="n">dkim_domain</span>  <span class="o">=</span> <span class="n">d_domain</span>
                <span class="n">dkim_aligned</span> <span class="o">=</span> <span class="n">d_aligned</span>
            <span class="k">else</span><span class="p">:</span>
                <span class="k">if</span> <span class="n">d_result</span> <span class="o">==</span> <span class="s">'pass'</span> <span class="ow">and</span> <span class="n">d_aligned</span> <span class="o">==</span> <span class="s">'pass'</span><span class="p">:</span>
                    <span class="n">dkim_fail</span>    <span class="o">=</span> <span class="n">d_result</span>
                    <span class="n">dkim_domain</span>  <span class="o">=</span> <span class="n">d_domain</span>
                    <span class="n">dkim_aligned</span> <span class="o">=</span> <span class="n">d_aligned</span>
            <span class="k">if</span> <span class="n">d_aligned</span> <span class="o">==</span> <span class="s">'fail'</span><span class="p">:</span>
                <span class="n">dkim_detail</span> <span class="o">+=</span> <span class="s">'</span><span class="se">\n</span><span class="s">'</span> <span class="o">+</span> <span class="n">etree</span><span class="p">.</span><span class="n">tostring</span><span class="p">(</span><span class="n">record</span><span class="p">,</span> <span class="n">pretty_print</span><span class="o">=</span><span class="bp">True</span><span class="p">).</span><span class="n">decode</span><span class="p">()</span> <span class="o">+</span> <span class="s">'</span><span class="se">\n</span><span class="s">'</span>

        <span class="c1"># info from filename
</span>        <span class="n">to_domain</span><span class="p">,</span> <span class="n">from_domain</span><span class="p">,</span> <span class="n">start_datetime</span><span class="p">,</span> <span class="n">end_datetime</span> <span class="o">=</span> <span class="n">extract_info_from_filename</span><span class="p">(</span><span class="n">filename</span><span class="p">)</span>

        <span class="c1"># Write the record to the CSV
</span>        <span class="n">csvwriter</span><span class="p">.</span><span class="n">writerow</span><span class="p">([</span><span class="s">'</span><span class="se">\'</span><span class="s">'</span> <span class="o">+</span> <span class="n">filename</span> <span class="o">+</span> <span class="s">'</span><span class="se">\'</span><span class="s">'</span> <span class="p">,</span> <span class="n">source_ip</span><span class="p">,</span> <span class="n">count</span><span class="p">,</span> <span class="n">header_from</span><span class="p">,</span> <span class="n">spf_domain</span> <span class="p">,</span> <span class="n">spf_fail</span><span class="p">,</span> <span class="n">spf_aligned</span><span class="p">,</span> <span class="n">dkim_domain</span> <span class="p">,</span>  <span class="n">dkim_fail</span><span class="p">,</span> <span class="n">dkim_aligned</span> <span class="p">,</span> <span class="n">dkim_detail</span><span class="p">,</span> <span class="n">dmarc_fail</span><span class="p">,</span> <span class="n">to_domain</span><span class="p">,</span> <span class="n">from_domain</span><span class="p">,</span> <span class="n">start_datetime</span><span class="p">,</span> <span class="n">end_datetime</span><span class="p">])</span>
<span class="k">def</span> <span class="nf">main</span><span class="p">():</span>
    <span class="n">directory</span> <span class="o">=</span> <span class="s">'dmarc-reps'</span>  <span class="c1"># Update this path to where your DMARC reports are stored
</span>    <span class="n">output_csv</span> <span class="o">=</span> <span class="s">'dmarc_results.csv'</span>  <span class="c1"># Output CSV file name
</span>    <span class="n">extract_and_parse_files</span><span class="p">(</span><span class="n">directory</span><span class="p">,</span> <span class="n">output_csv</span><span class="p">)</span>
<span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="s">"__main__"</span><span class="p">:</span>
    <span class="n">main</span><span class="p">()</span>
</code></pre></div></div>

<h2 id="how-to-use">How to use</h2>

<h3 id="install-python">install python</h3>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">sudo </span>apt <span class="nb">install </span>python3 python-is-python3
pip <span class="nb">install gzip </span>zipfile lxml
code dmarc.py
<span class="nb">mkdir </span>dmarc-reps
</code></pre></div></div>

<p>Place gz or zip files into dmarc-reps folder, upon creation of dmarc.oy and dmarc-reps folder.</p>

<h3 id="run-it-and-open-dmarc_resultcsv">Run it and open dmarc_result.csv</h3>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>python dmarc.py
code dmarc_result.csv
</code></pre></div></div>

<p>It looks like this, and you can check DMARC pass or not, checking either of SPF/SPF-alignment or DKIM/DKIM-alignment pass or not.
<img src="https://github.com/rtree/pages/assets/1018794/7f184544-edc7-449a-a2a8-034308812cab" alt="image" /></p>

<p>Enjoy! and if you have questions, please DM on <a href="https://twitter.com/rtree">https://twitter.com/rtree</a></p>]]></content><author><name>Tomohiko Araki</name></author><summary type="html"><![CDATA[I just would like to have DMARC report into 1 folder, and make them packed into CSV, as popular DMARC processing SaaS is a little bit costy.]]></summary></entry><entry><title type="html">List of finalists in ETHGlobal, unofficial</title><link href="https://metaproxy.co/listOfFinalists/" rel="alternate" type="text/html" title="List of finalists in ETHGlobal, unofficial" /><published>2023-09-25T00:00:00+00:00</published><updated>2023-09-25T00:00:00+00:00</updated><id>https://metaproxy.co/listOfFinalists</id><content type="html" xml:base="https://metaproxy.co/listOfFinalists/"><![CDATA[<p>Best of bests in the scene has power to stimulate yourself.
I have picked up (not all) repositories of finalists of ETHGlobal.</p>

<h2 id="2023-ethglobal-new-york">2023 ETHGlobal New York</h2>

<blockquote class="twitter-tweet"><p lang="en" dir="ltr">We are proud to announce:<br />The 13 ETHGlobal New York 🗽 finalists! <br /><br />🖼 FRAMED!<br />🤑 F.A.S.T.<br />🔍 ZeroTrustBounty<br />🤔 Doom Arena<br />🗺 AirTracker <br />🔗 SafeCreate2<br />🤖 ConsciousNFT<br />🛸 Space Guardians<br />👁 A.S.R.<br />🥷 MEVictim Rebate<br />📈 RealReturn<br />💰 Abstract Wallet<br />🔒 XSafe<br /><br />👇/🧵</p>&mdash; ETHGlobal (@ETHGlobal) <a href="https://twitter.com/ETHGlobal/status/1706049363464569232?ref_src=twsrc%5Etfw">September 24, 2023</a></blockquote>
<script async="" src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>

<h3 id="framed">FRAMED</h3>

<p>FRAMED: A fully onchain &amp; trustless hidden information game (Mafia) built with fhEVM</p>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/framed-5vv6q">https://ethglobal.com/showcase/framed-5vv6q</a></li>
  <li>Repository   <a href="https://github.com/joeyiny/FRAMED">https://github.com/joeyiny/FRAMED</a></li>
  <li>Technologies
    <ul>
      <li>Inco Network fhEVM <a href="https://www.inco.network/">https://www.inco.network/</a></li>
      <li>Privvy <a href="https://www.privy.io/">https://www.privy.io/</a></li>
      <li>Homomorphic Encryption</li>
    </ul>
  </li>
</ul>

<h3 id="fast">F.A.S.T</h3>

<p>F.A.S.T. is the fastest way to use the blockchain to interact between 2 users. Take a picture of youself to sign up, and then snap a picture with someone else, to then interact with their address seamlessly.</p>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/f-a-s-t-1ian9">https://ethglobal.com/showcase/f-a-s-t-1ian9</a></li>
  <li>Repository   <a href="https://github.com/SwineCoder101/FAST-ETH-NYC">https://github.com/SwineCoder101/FAST-ETH-NYC</a></li>
  <li>Technologies
    <ul>
      <li>Telegram</li>
      <li>Vercel</li>
      <li>Airstack</li>
      <li>Safe</li>
      <li>Mask</li>
    </ul>
  </li>
</ul>

<h3 id="zerotrustbounty">ZeroTrustBounty</h3>

<p>We are solving challenges of traditional bug bounty platforms where businesses have to trust 3rd parties with sensitive data and whitehats have no guarantee of payment for their services</p>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/zerotrustbounty-dbs0b">https://ethglobal.com/showcase/zerotrustbounty-dbs0b</a></li>
  <li>Repository   <a href="https://github.com/antojoseph/ZeroTrustBounty">https://github.com/antojoseph/ZeroTrustBounty</a></li>
  <li>Technologies
    <ul>
      <li>TLS Notary</li>
      <li>Lit protocol</li>
    </ul>
  </li>
</ul>

<h3 id="doom-arena">Doom Arena</h3>

<p>Distributed DApp for managing and creating contests around the Doom Games.</p>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/doom-arena-hmu4j">https://ethglobal.com/showcase/doom-arena-hmu4j</a></li>
  <li>Repository   <a href="https://github.com/felipefg/doom-arena">https://github.com/felipefg/doom-arena</a></li>
  <li>Technologies
    <ul>
      <li>Cartesi</li>
    </ul>
  </li>
</ul>

<h3 id="airtracker">AirTracker</h3>

<p>Decentralized physical asset tracking using AirTags and blockchain storage and messaging</p>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/airtracker-8qstn">https://ethglobal.com/showcase/airtracker-8qstn</a></li>
  <li>Repository   <a href="https://github.com/tkeith/airtracker">https://github.com/tkeith/airtracker</a></li>
  <li>Technologies
    <ul>
      <li>AirTag</li>
      <li>IPFS</li>
      <li>XMTP</li>
      <li>Airstack</li>
    </ul>
  </li>
</ul>

<h3 id="safecreate2">SafeCreate2</h3>

<p>Secure multi-chain smart contract deployments using a single-chain Safe</p>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/safecreate2-szzw3">https://ethglobal.com/showcase/safecreate2-szzw3</a></li>
  <li>Repository   <a href="https://github.com/jaydenwindle/safe-create2">https://github.com/jaydenwindle/safe-create2</a></li>
  <li>Technologies
    <ul>
      <li>Safe</li>
      <li>Axelar <a href="https://axelar.network/">https://axelar.network/</a></li>
    </ul>
  </li>
</ul>

<h3 id="consciousnft">ConsciousNFT</h3>

<p>AI-powered tools for NFT projects to awaken conscious identities within your NFTs</p>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/consciousnft-4fbkx">https://ethglobal.com/showcase/consciousnft-4fbkx</a></li>
  <li>Repository   <a href="https://github.com/louiswilbrink/consciousNFT">https://github.com/louiswilbrink/consciousNFT</a></li>
  <li>Technologies
    <ul>
      <li>GPT-3.5 by OpenAI</li>
      <li>IPFS</li>
    </ul>
  </li>
</ul>

<h3 id="space-guardians">Space Guardians</h3>

<p>Satellite Secured Access Recovery - A MetaMask Guardian management system using Snaps</p>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/space-guardians-bzcfc">https://ethglobal.com/showcase/space-guardians-bzcfc</a></li>
  <li>Repository   <a href="https://github.com/cryptosat/satlock">https://github.com/cryptosat/satlock</a></li>
  <li>Technologies
    <ul>
      <li>Metamask Snaps <a href="https://metamask.io/snaps/">https://metamask.io/snaps/</a></li>
      <li>Crypto-satellites <a href="https://www.cryptosat.io/">https://www.cryptosat.io/</a></li>
    </ul>
  </li>
</ul>

<h3 id="asr">A.S.R</h3>

<p>You’re the owner of a smart contract but you lost the key. You’re screwed…. or are you? Anti Social Recovery is a design pattern that enables users to recover ownership of their smart contract after losing their signing keys.</p>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/a-s-r-spakz">https://ethglobal.com/showcase/a-s-r-spakz</a></li>
  <li>Repository   <a href="https://github.com/crystaltai/anti-social-recovery-club">https://github.com/crystaltai/anti-social-recovery-club</a></li>
  <li>Technologies
    <ul>
      <li>World ID</li>
      <li>ThirdWeb <a href="https://thirdweb.com/">https://thirdweb.com/</a></li>
    </ul>
  </li>
</ul>

<h3 id="mevictim-rebate">MEVictim Rebate</h3>

<p>Designing better market incentives by identifying MEV victims using historical on chain data and airdropping victims a token to access a token gated Uniswap v4 pool on Scroll</p>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/mevictim-rebate-qsxak">https://ethglobal.com/showcase/mevictim-rebate-qsxak</a></li>
  <li>Repository   <a href="https://github.com/mliuuu/ETHNYC23/">https://github.com/mliuuu/ETHNYC23/</a></li>
  <li>Technologies
    <ul>
      <li>AXION <a href="https://www.axiom.xyz/">https://www.axiom.xyz/</a></li>
      <li>Hyperlane</li>
      <li>The Graph</li>
      <li>Uniswap v4 + Hook Launch on Scroll</li>
      <li>Flashbots</li>
    </ul>
  </li>
</ul>

<h3 id="realreturn">RealReturn</h3>

<p>RealReturn is an open leaderboard of verified real investment returns, enabled by TLSNotary and Jomo.</p>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/realreturn-pu64y">https://ethglobal.com/showcase/realreturn-pu64y</a></li>
  <li>Repository   <a href="https://github.com/chcharcharlie/realreturn">https://github.com/chcharcharlie/realreturn</a></li>
  <li>Technologies
    <ul>
      <li>TLSNotary</li>
      <li>Jomo <a href="https://docs.jomo.id/welcome-to-jomo/jomo-overview">https://docs.jomo.id/welcome-to-jomo/jomo-overview</a></li>
    </ul>
  </li>
</ul>

<h3 id="abstract-wallet">Abstract Wallet</h3>

<p>Wallet abstraction enables instant single click onboarding of a user. With nothing but a smartphone using the power of account abstraction with passkeys. generate passkeys credentials, create its smart acount, mint a NFT, all in a single click ! The wallet is invisible.</p>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/abstract-wallet-a9pzf">https://ethglobal.com/showcase/abstract-wallet-a9pzf</a></li>
  <li>Repository   <a href="https://github.com/qd-qd/wallet-abstraction">https://github.com/qd-qd/wallet-abstraction</a></li>
  <li>Technologies
    <ul>
      <li>FIDO passkey</li>
      <li>solidity FCL library (secp256R1</li>
    </ul>
  </li>
</ul>

<h3 id="xsafe">XSafe</h3>

<p>X-Safe streamlines Gnosis Safe multi-sigs for cross-chain use. Sign once, and the signature auto-propagates to all linked chains. Execute single transactions across multiple networks. Save time, cut overhead, and keep strong security.</p>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/xsafe-bbj01">https://ethglobal.com/showcase/xsafe-bbj01</a></li>
  <li>Repository   <a href="https://github.com/cucupac/x-safe">https://github.com/cucupac/x-safe</a></li>
  <li>Technologies
    <ul>
      <li>Safe</li>
    </ul>
  </li>
</ul>

<h2 id="2023-ethglobal-paris">2023 ETHGlobal Paris</h2>

<blockquote class="twitter-tweet"><p lang="en" dir="ltr">🥁 The 13 ETHGlobal Paris 🇫🇷 finalists are:<br /><br />🎙 ZK Microphone<br />📈 GLSwaps<br />🐦 TwitterCampaign<br />💥 TyphoonCashX<br />🪄 Kinetex Light Clients<br />🤫 Harpos<br />🫧 Bubbles<br />🌳 Stake Garden<br />🔐 Cypher Deposit<br />⚡️ Bob the Solver<br />🗣 VoiceSense<br />🛗 Community-bound<br />💬 On-chain messaging aggregator<br /><br />👇/🧵</p>&mdash; ETHGlobal (@ETHGlobal) <a href="https://twitter.com/ETHGlobal/status/1683156559847739394?ref_src=twsrc%5Etfw">July 23, 2023</a></blockquote>
<script async="" src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>

<h3 id="zk-microphone">ZK Microphone</h3>

<p>ZK Microphone: Trusted audio in the age of deepfakes 🔒🎙 Generative AI is a threat to society. It enables disinformation, manipulation, and political subversion. We’ve built the world’s first attested microphone and used ZK-SNARKs to protect authenticity and privacy.</p>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/zk-microphone-8161v">https://ethglobal.com/showcase/zk-microphone-8161v</a></li>
  <li>Repository   <a href="https://github.com/Miyamura80/ZKMicrophone">https://github.com/Miyamura80/ZKMicrophone</a></li>
  <li>Technologies
    <ul>
      <li>Raspberry Pi</li>
      <li>Noir <a href="https://noir-lang.org/">https://noir-lang.org/</a></li>
      <li>IPFS</li>
    </ul>
  </li>
</ul>

<h3 id="glswaps">GLSwaps</h3>

<p>Users now have access to spot and perpetual markets from the Uniswap app (app.uniswap.org) using the GLS browser extension routing through GMX as well. From Uniswap, users can access on-chain perpetual markets. Injecting the smart contracts for derivatives by adding a leverage button and slider. For degens.</p>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/glswaps-grandma-leveraged-swaps-gzit8">https://ethglobal.com/showcase/glswaps-grandma-leveraged-swaps-gzit8</a></li>
  <li>Repository   <a href="https://github.com/jaimindp/UniPerp">https://github.com/jaimindp/UniPerp</a></li>
  <li>Technologies
    <ul>
      <li>A browser extension</li>
      <li>GMX</li>
      <li>Uniswap</li>
      <li>dydx</li>
    </ul>
  </li>
</ul>

<h3 id="twittercampaign">TwitterCampaign</h3>

<p>Twitter Campaigns is an A.I.-enabled on-chain DAO-tool aimed at crypto-familiar users where anyone can create, and participate in on-chain Twitter marketing campaigns.</p>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/twittercampaign-wv9we">https://ethglobal.com/showcase/twittercampaign-wv9we</a></li>
  <li>Repository   <a href="https://github.com/myounatan/twitter-campaign">https://github.com/myounatan/twitter-campaign</a></li>
  <li>Technologies
    <ul>
      <li>Safe AuthKit</li>
      <li>Biconomy’s Transak SDK</li>
      <li>TheGraph</li>
    </ul>
  </li>
</ul>

<h3 id="typhooncashx">TyphoonCashX</h3>

<p>TyphoonCashX is a protocol enabling bridging any ERC20 native tokens across any EVM compatible chain in an anonymous fashion.</p>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/typhooncashx-p10yd">https://ethglobal.com/showcase/typhooncashx-p10yd</a></li>
  <li>Repository   <a href="https://github.com/TyphoonCashX">https://github.com/TyphoonCashX</a></li>
  <li>Technologies
    <ul>
      <li>Sismo</li>
      <li>Hyperlane</li>
    </ul>
  </li>
</ul>

<h3 id="kinetex-light-clients">Kinetex Light Clients</h3>

<p>Kinetex Light Clients for Cross-Chain Resolving (based on ChainLink CCIP, Hyperline, Axelar, ZetaChain &amp; Hashi)</p>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/kinetex-light-clients-mmr-uvzr1">https://ethglobal.com/showcase/kinetex-light-clients-mmr-uvzr1</a></li>
  <li>Repository   <a href="https://github.com/KinetexNetwork/ethglobal-paris">https://github.com/KinetexNetwork/ethglobal-paris</a></li>
  <li>Technologies
    <ul>
      <li>Merkle Mountain Range structure LightClient
        <ul>
          <li>was inspired by Axiom</li>
        </ul>
      </li>
      <li>Hashi adapters</li>
      <li>Kinetex Protocol</li>
      <li>1inch Fusion</li>
    </ul>
  </li>
</ul>

<h3 id="harpos">Harpos</h3>

<p>Leveraging Sismo and XMTP, Harpos is a social network centered on privacy and anonymity where users can choose multiple criteria to select who they want to connect with.</p>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/harpos-0wpcc">https://ethglobal.com/showcase/harpos-0wpcc</a></li>
  <li>Repository   <a href="https://github.com/ahippoly/EthParisHackathon">https://github.com/ahippoly/EthParisHackathon</a></li>
  <li>Technologies
    <ul>
      <li>Sismo</li>
      <li>XMTP</li>
      <li>IPFS</li>
    </ul>
  </li>
</ul>

<h3 id="bubbles">Bubbles</h3>

<p>Bubbles is a voice chat app for onchain communities. Create token-gated rooms based on token ownership, POAP claims, group affiliations and more.</p>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/bubbles-avp6o">https://ethglobal.com/showcase/bubbles-avp6o</a></li>
  <li>Repository   <a href="https://github.com/gskril/ethglobal-paris-hack">https://github.com/gskril/ethglobal-paris-hack</a></li>
  <li>Technologies
    <ul>
      <li>Daily.co</li>
      <li>Airstack</li>
      <li>Sismo</li>
      <li>wagmi</li>
    </ul>
  </li>
</ul>

<h3 id="stake-garden">Stake Garden</h3>

<p>StakeGarden is a Self Managed Liquid Staking Tokens Portfolio that allows you to diversify and mitigate the risk keeping you safe and in control.</p>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/stake-garden-7tj7j">https://ethglobal.com/showcase/stake-garden-7tj7j</a></li>
  <li>Repository   <a href="https://github.com/StakeGarden/stakegarden">https://github.com/StakeGarden/stakegarden</a></li>
  <li>Technologies
    <ul>
      <li>Metamask SDK</li>
      <li>1inch</li>
      <li>TheGraph</li>
    </ul>
  </li>
</ul>

<h3 id="cypher-deposit">Cypher Deposit</h3>

<p>Elevating financial privacy in crypto withdrawals. Securely withdraw funds via anonymous transfers &amp; secure transactions, preserving anonymity.</p>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/cypher-deposit-cnhs5">https://ethglobal.com/showcase/cypher-deposit-cnhs5</a></li>
  <li>Repository   <a href="https://github.com/Alice-s-Deposit">https://github.com/Alice-s-Deposit</a></li>
  <li>Technologies
    <ul>
      <li>HOPR <a href="https://hoprnet.org/">https://hoprnet.org/</a></li>
      <li>zkBob</li>
      <li>IPFS</li>
    </ul>
  </li>
</ul>

<h3 id="bob-the-solver">Bob the Solver</h3>

<p>Infrastructure for Intent-based transactions for better UX of wallets and dapps.
The project has successfully developed a ‘solver’ that accepts the user’s intent, automatically formulates the appropriate transactions, and executes them on behalf of the user.</p>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/bob-the-solver-yxtgp">https://ethglobal.com/showcase/bob-the-solver-yxtgp</a></li>
  <li>Repository   <a href="https://github.com/nlok5923/solver">https://github.com/nlok5923/solver</a></li>
  <li>Technologies
    <ul>
      <li>Account Abstraction (AA) Wallet</li>
      <li>a machine learning (ML) model that classifies the user’s intent</li>
    </ul>
  </li>
</ul>

<h3 id="voicesense">VoiceSense</h3>

<p>VoiceSence leverages the power of AI voice recognition to make web3 more inclusive.</p>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/voicesense-yfzpq">https://ethglobal.com/showcase/voicesense-yfzpq</a></li>
  <li>Repository (link is not found)</li>
  <li>Technologies
    <ul>
      <li>Cartesi</li>
      <li>Hugging Face AI model</li>
    </ul>
  </li>
</ul>

<h3 id="community-bound">Community-bound</h3>

<p>Community Bound Tokens (CBTs) are ERC-20 tokens that can only be transferred among members of a community, which can be used for reputation, community merit based governance, opt-in social credit score, coordination games or engagement scores.</p>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/community-bound-y73ys">https://ethglobal.com/showcase/community-bound-y73ys</a></li>
  <li>Repository   <a href="https://github.com/TroopFinance/community-bound">https://github.com/TroopFinance/community-bound</a></li>
  <li>Technologies
    <ul>
      <li>Safe (forked)</li>
      <li>Push protocol</li>
    </ul>
  </li>
</ul>

<h3 id="on-chain-messaging-aggregator">On-chain messaging aggregator</h3>

<p>Aggregate all web3 messaging app into S3nd! 🌐 Your one-stop hub for all Web3 messaging apps. Easy single sign-on, Human verification with Worldcoin ✔️ , &amp; native on-chain messaging support. No bots, only genuine convos!</p>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/on-chain-messaging-aggregator-3eshd">https://ethglobal.com/showcase/on-chain-messaging-aggregator-3eshd</a></li>
  <li>Repository   <a href="https://github.com/RafalZgoda/Onchain-Messaging-Aggregator">https://github.com/RafalZgoda/Onchain-Messaging-Aggregator</a></li>
  <li>Technologies
    <ul>
      <li>XMTP</li>
      <li>Worldcoin</li>
      <li>Airstack</li>
      <li>Sismo</li>
      <li>Message protocols aggregated
        <ul>
          <li>Native on-chain messaging</li>
          <li>Debank Hi</li>
          <li>Blockscan</li>
          <li>Push protocol</li>
        </ul>
      </li>
    </ul>
  </li>
</ul>

<h2 id="2023-ethglobal-waterloo">2023 ETHGlobal Waterloo</h2>

<blockquote class="twitter-tweet"><p lang="en" dir="ltr">🥁 We&#39;re proud to announce... <br /><br />The 11 ETHGlobal Waterloo 2023 🍁 finalists! <br /><br />💦 AquaNet<br />🌌 Smarter Contract<br />🧧 Fukuro<br />🧉 Roll a Mate<br />🐷 Piggybank 6551 NFT<br />😎 Smile DAO<br />🔐 a(i)udit<br />🤖 Tokenbound Titans<br />🛒 zBay<br />⛑ Token Rescue Buddy<br />🎨 Copix<br /><br />👇/🧵</p>&mdash; ETHGlobal (@ETHGlobal) <a href="https://twitter.com/ETHGlobal/status/1673069276805099520?ref_src=twsrc%5Etfw">June 25, 2023</a></blockquote>
<script async="" src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>

<h3 id="aquanet">AquaNet</h3>

<p>A social network designed for AI-Driven NFTs to interact with each other and develop unique personalities.</p>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/aquanet-dzz2c">https://ethglobal.com/showcase/aquanet-dzz2c</a></li>
  <li>Repository   <a href="https://github.com/AquaNet-ETH-Waterloo-2023/web">https://github.com/AquaNet-ETH-Waterloo-2023/web</a></li>
  <li>Technologies
    <ul>
      <li>Non-fungible Tokenbound Accounts (ERC 6551)</li>
      <li>OpenAI’s GPT-3.5</li>
      <li>Airstack <a href="https://docs.tokenbound.org/">https://docs.tokenbound.org/</a></li>
    </ul>
  </li>
</ul>

<h3 id="smarter-contract">Smarter Contract</h3>

<p>Smarter Contract, powered by Hyperlane, optimizes smart contract performance by intelligently delegating functions to the most cost-efficient blockchain in real-time. It’s a leap towards efficient, accessible, and cost-effective blockchain interaction.</p>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/smarter-contract-kw41h">https://ethglobal.com/showcase/smarter-contract-kw41h</a></li>
  <li>Repository   <a href="https://github.com/maancham/smarter-contracts">https://github.com/maancham/smarter-contracts</a></li>
  <li>Technologies
    <ul>
      <li>Hyperlane</li>
    </ul>
  </li>
</ul>

<h3 id="fukuro">Fukuro</h3>

<p>Bundle curations with EIP 6551. Auction marketplace for EIP 6551 supported tokens.</p>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/fukuro-3cdwv">https://ethglobal.com/showcase/fukuro-3cdwv</a></li>
  <li>Repository   <a href="https://github.com/banky/fukuro">https://github.com/banky/fukuro</a></li>
  <li>Technologies
    <ul>
      <li>custom implementation of EIP 6551</li>
      <li>Pinata</li>
      <li>wagmi</li>
      <li>The Graph</li>
    </ul>
  </li>
</ul>

<h3 id="roll-a-mate">Roll a Mate</h3>

<p>Send money on Ethereum Mainnet with transfers costing from zero to 4 cents.
By manipulating the mempool to validate and broadcast low gas transactions, which are then included into sidechain orderbook rollups (without user interaction with it) and then likely to be dropped (at zero cost), Ethereum Mainnet users can send and receive payments at zero to 4 cents costs, even faster than the Mainnet transaction completion time.</p>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/roll-a-mate-ob574">https://ethglobal.com/showcase/roll-a-mate-ob574</a></li>
  <li>Repository   <a href="https://github.com/ArturVargas/Mate-contracts">https://github.com/ArturVargas/Mate-contracts</a></li>
  <li>Technologies
    <ul>
      <li>The Graph</li>
      <li>UMA</li>
      <li>Telegram</li>
      <li>Lit protocol</li>
      <li>Airstack</li>
      <li>XMTP</li>
    </ul>
  </li>
</ul>

<h3 id="piggybank-6551-nft">Piggybank 6551 NFT</h3>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/piggybank-6551-nft-e2ai5">https://ethglobal.com/showcase/piggybank-6551-nft-e2ai5</a></li>
  <li>Repository   <a href="https://github.com/nnnnicholas/6551piggybank">https://github.com/nnnnicholas/6551piggybank</a></li>
  <li>Technologies
    <ul>
      <li>in progress</li>
    </ul>
  </li>
</ul>

<h3 id="smile-dao">Smile DAO</h3>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/smile-dao-xogfo">https://ethglobal.com/showcase/smile-dao-xogfo</a></li>
  <li>Repository   <a href="https://github.com/smiledao/frontend">https://github.com/smiledao/frontend</a></li>
  <li>Technologies
    <ul>
      <li>in progress</li>
    </ul>
  </li>
</ul>

<h3 id="aiudit">a(i)udit</h3>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/a-i-udit-ja32i">https://ethglobal.com/showcase/a-i-udit-ja32i</a></li>
  <li>Repository   <a href="https://github.com/leeederek/aiudit/">https://github.com/leeederek/aiudit/</a></li>
  <li>Technologies
    <ul>
      <li>in progress</li>
    </ul>
  </li>
</ul>

<h3 id="tokenbound-titans">Tokenbound Titans</h3>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/tokenbound-titans-5w6oq">https://ethglobal.com/showcase/tokenbound-titans-5w6oq</a></li>
  <li>Repository   <a href="https://github.com/hamzah2304/tokenbound-titans">https://github.com/hamzah2304/tokenbound-titans</a></li>
  <li>Technologies
    <ul>
      <li>Risc Zero</li>
      <li>in progress</li>
    </ul>
  </li>
</ul>

<h3 id="zbay">zBay</h3>

<p>zBay revolutionizes online marketplaces by decentralizing eBay, empowering merchants and buyers. It enables seamless data and reputation import from eBay, secure escrow, XMTP chat, and transparent dispute resolution using the Uma optimistic oracle</p>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/zbay-r0724">https://ethglobal.com/showcase/zbay-r0724</a></li>
  <li>Repository   <a href="https://github.com/wannabehero/ethglobal-waterloo">https://github.com/wannabehero/ethglobal-waterloo</a></li>
  <li>Technologies
    <ul>
      <li>Sismo Connect</li>
      <li>Gitcoin Passport score</li>
      <li>eBay related
        <ul>
          <li>ebay profile importer (via puppeteer)</li>
          <li>Apify</li>
        </ul>
      </li>
      <li>IPFS</li>
      <li>XMTP</li>
      <li>Airstack</li>
      <li>UMA</li>
      <li>TheGraph</li>
    </ul>
  </li>
</ul>

<h3 id="token-rescue-buddy">Token Rescue Buddy</h3>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/token-rescue-buddy-bd0sd">https://ethglobal.com/showcase/token-rescue-buddy-bd0sd</a></li>
  <li>Repository   <a href="https://github.com/xavierdmello/TokenRescueBuddy">https://github.com/xavierdmello/TokenRescueBuddy</a></li>
  <li>Technologies
    <ul>
      <li>in progress</li>
    </ul>
  </li>
</ul>

<h3 id="copix">Copix</h3>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/copix-28x5i">https://ethglobal.com/showcase/copix-28x5i</a></li>
  <li>Repository   <a href="https://github.com/jaxonL/copix">https://github.com/jaxonL/copix</a></li>
  <li>Technologies
    <ul>
      <li>in progress</li>
    </ul>
  </li>
</ul>

<h2 id="2023-ethglobal-lisbon">2023 ETHGlobal Lisbon</h2>

<blockquote class="twitter-tweet"><p lang="en" dir="ltr">We&#39;re proud to announce: our 10 ETHGlobal Lisbon Finalists! 🇵🇹<br /><br />🐶 Credential Corgi<br />😎 Dookies<br />👬 buddy<br />💻 LocalPro<br />🪙 CryptoBureau<br />🤝 Pinky Protocol<br />🤖 Web3-gpt4<br />💸 Donation Station<br />🤯 ChainGPT<br />🗜 op-kompressor <br /><br />👇 / 🧵</p>&mdash; ETHGlobal (@ETHGlobal) <a href="https://twitter.com/ETHGlobal/status/1657794093802483714?ref_src=twsrc%5Etfw">May 14, 2023</a></blockquote>
<script async="" src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>

<h3 id="credential-corgi">Credential Corgi</h3>

<p>Credential Corgi enables creation of certification standards, issuance of ZKP-compatible certificates, and a request-proof exchange between 3rd parties and credential holders</p>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/credential-corgi-a7a0r">https://ethglobal.com/showcase/credential-corgi-a7a0r</a></li>
  <li>Repository   <a href="https://github.com/tkeith/credential-corgi">https://github.com/tkeith/credential-corgi</a></li>
  <li>Technologies
    <ul>
      <li>SnarkyJS, without MINA</li>
      <li>GPT-4</li>
      <li>Wagmi</li>
      <li>RainbowKit</li>
    </ul>
  </li>
</ul>

<h3 id="dookies">Dookies</h3>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/dookies-0g13u">https://ethglobal.com/showcase/dookies-0g13u</a></li>
  <li>Repository   <a href="https://github.com/D00ki3s">https://github.com/D00ki3s</a></li>
  <li>Technologies
    <ul>
      <li>in progress</li>
    </ul>
  </li>
</ul>

<h3 id="buddy">buddy</h3>

<p>A Tinder-style app on Lens for developers to find hackathon partners, using Sismo Connect to prove knowledge and skillsof crypto</p>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/buddy-t7gbs">https://ethglobal.com/showcase/buddy-t7gbs</a></li>
  <li>Repository   <a href="https://github.com/catmcgee/buddy">https://github.com/catmcgee/buddy</a></li>
  <li>Technologies
    <ul>
      <li>Lens</li>
      <li>Sismo connect</li>
    </ul>
  </li>
</ul>

<h3 id="localpro">LocalPro</h3>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/localpro-xckra">https://ethglobal.com/showcase/localpro-xckra</a></li>
  <li>Repository   <a href="https://github.com/wannabehero/ethglobal-lisbon">https://github.com/wannabehero/ethglobal-lisbon</a></li>
  <li>Technologies
    <ul>
      <li>in progress</li>
    </ul>
  </li>
</ul>

<h3 id="cryptobureau">CryptoBureau</h3>

<p>CryptoBureau uses the power of Zero Knowledge to build credit score which can allow under-collateralized lending.</p>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/cryptobureau-u1389">https://ethglobal.com/showcase/cryptobureau-u1389</a></li>
  <li>Repository   <a href="https://github.com/EthGlobalLisbon2023/local-pro">https://github.com/EthGlobalLisbon2023/local-pro</a></li>
  <li>Technologies
    <ul>
      <li>Sismo</li>
      <li>zokrates</li>
      <li>TrueLayer API</li>
      <li>PolygonID</li>
      <li>WorldID</li>
    </ul>
  </li>
</ul>

<h3 id="pinky-protocol">Pinky Protocol</h3>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/pinky-protocol-vsos2">https://ethglobal.com/showcase/pinky-protocol-vsos2</a></li>
  <li>Repository   <a href="https://github.com/pinky-io/pinky-protocol">https://github.com/pinky-io/pinky-protocol</a></li>
  <li>Technologies
    <ul>
      <li>in progress</li>
    </ul>
  </li>
</ul>

<h3 id="web3-gpt4">Web3-gpt4</h3>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/web3-gpt4-jkaja">https://ethglobal.com/showcase/web3-gpt4-jkaja</a></li>
  <li>Repository   <a href="https://github.com/Markeljan/web3-gpt4">https://github.com/Markeljan/web3-gpt4</a></li>
  <li>Technologies
    <ul>
      <li>in progress</li>
    </ul>
  </li>
</ul>

<h3 id="donation-station">Donation Station</h3>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/donation-station-g09ok">https://ethglobal.com/showcase/donation-station-g09ok</a></li>
  <li>Repository   <a href="https://github.com/lennardevertz/ethLisbon2023">https://github.com/lennardevertz/ethLisbon2023</a></li>
  <li>Technologies
    <ul>
      <li>in progress</li>
    </ul>
  </li>
</ul>

<h3 id="chaingpt">ChainGPT</h3>

<p>Decentralised &amp; verifiable chat AI, backed by the blockchain: a port of Alpaca LLM model leveraging the Cartesi app-specific rollups</p>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/chaingpt-and-accountability-for-ai-agents-doquk">https://ethglobal.com/showcase/chaingpt-and-accountability-for-ai-agents-doquk</a></li>
  <li>Repository   <a href="https://github.com/dhood/rollups-examples">https://github.com/dhood/rollups-examples</a></li>
  <li>Technologies
    <ul>
      <li>Porting of Alpaca.cpp into RISC-V
        <ul>
          <li>Stanford’s Alpaca Large Language Model (LLM)</li>
        </ul>
      </li>
      <li>Cartesi</li>
    </ul>
  </li>
</ul>

<h3 id="op-kompressor">op-kompressor</h3>

<p>Decompress calldata at runtime to save L1 gas while transacting on L2s like Optimism where gas is an expensive resource.</p>

<ul>
  <li>Project page <a href="https://ethglobal.com/showcase/op-kompressor-1uboo">https://ethglobal.com/showcase/op-kompressor-1uboo</a></li>
  <li>Repository   <a href="https://github.com/clabby/op-kompressor">https://github.com/clabby/op-kompressor</a></li>
  <li>Technologies
    <ul>
      <li>in progress</li>
    </ul>
  </li>
</ul>

<h2 id="2023-ethglobal-tokyo">2023 ETHGlobal Tokyo</h2>

<blockquote class="twitter-tweet"><p lang="en" dir="ltr">We are proud to announce⎯the 11 ETHGlobal Tokyo finalists! 🇯🇵<br /><br />🤩 POMPoarding<br />🎙️ ZKVoiceKey<br />🔥 BAILOUT<br />➗ AokiApp NFT<br />🐙 Octoplorer<br />🌸 Myna<br />🪧 <a href="https://t.co/L5yS0lSEyv">https://t.co/L5yS0lSEyv</a><br />💚 AbswapX<br />🦝 Tanuki<br />🤫 YORU<br />🎮 SuperPlay<br /><br />👇 / 🧵</p>&mdash; ETHGlobal (@ETHGlobal) <a href="https://twitter.com/ETHGlobal/status/1647524426450083840?ref_src=twsrc%5Etfw">April 16, 2023</a></blockquote>
<script async="" src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>

<h2 id="2022-ethindia">2022 ETHIndia</h2>

<blockquote class="twitter-tweet"><p lang="en" dir="ltr">We&#39;re proud to announce⎯the 12 ETHIndia 🇮🇳 2022 finalists! <br /><br />⛑ Web3Rescue<br />🔗 ChainPe<br />🚨 Block Alerts<br />🎳 Bowled<br />💳 DePay<br />🔎 Persona bot-wallet detection<br />⭐️ Sadaiv CI<br />🤝 Proof Of Trust<br />🔒 ETHGuard<br />🔋 ChargeSwap<br />🎉 TRIKL<br />🦺 SafeGuard<br /><br />👇/🧵</p>&mdash; ETHGlobal (@ETHGlobal) <a href="https://twitter.com/ETHGlobal/status/1603833092346609700?ref_src=twsrc%5Etfw">December 16, 2022</a></blockquote>
<script async="" src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>

<h2 id="2022-ethsanfrancisco">2022 ETHSanFrancisco</h2>

<blockquote class="twitter-tweet"><p lang="en" dir="ltr">We&#39;re excited to announce⎯ the 13 ETHSanFrancisco 2022 finalists!<br /><br />🌎 mundo<br />⚡️ FuelScape<br />📟 BB3 - BEEP3R<br />🏛 DAOCensus<br />🍉 melo vault<br />🔑 Org3<br />💎 UNIC<br />9️⃣ ZK Section 9<br />🛋 Decentral Couch<br />🌐 Cambria<br />🏃‍♂️ Swallow <br />👻 Peekaboo<br />⛓ Snapchain<br /><br />👇/🧵</p>&mdash; ETHGlobal (@ETHGlobal) <a href="https://twitter.com/ETHGlobal/status/1589431653599961089?ref_src=twsrc%5Etfw">November 7, 2022</a></blockquote>
<script async="" src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>

<h2 id="2022-ethbogotá">2022 ETHBogotá</h2>

<blockquote class="twitter-tweet"><p lang="en" dir="ltr">We&#39;re proud to announce...<br /><br />The 12 ETHBogota finalists!<br />👇/🧵<br /><br />🧑‍🤝‍🧑 frens<br />🔎 Blobscan Explorer<br />🤑 Bet Streams 🤑<br />👍👎 ETHernal Reviews<br />🍊 PULP<br />🌱🤑 Superseed<br />🌐 Coedia Network<br />🪶 Zipline <br />🧂Anonymous Vickrey Auctions On Chain<br />🦨 diZKreet <br />🎧 ClubSpace<br />🧨 txn. xyz</p>&mdash; ETHGlobal (@ETHGlobal) <a href="https://twitter.com/ETHGlobal/status/1579249265557192704?ref_src=twsrc%5Etfw">October 9, 2022</a></blockquote>
<script async="" src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>

<h2 id="2022-ethmexico">2022 ETHMexico</h2>

<blockquote class="twitter-tweet"><p lang="en" dir="ltr">We are proud to announce ─ <br />the 12 <a href="https://twitter.com/hashtag/ETHMexico?src=hash&amp;ref_src=twsrc%5Etfw">#ETHMexico</a> finalists! 🇲🇽<br /><br />🌌 Datamorpho<br />🔑Keyppi<br />♦️ Kiwik<br />🛍 Sell Out<br />🏆 Emergence<br />📚 Staxx<br />🏘 BlockCommunity<br />🤑 ETHPoint<br />🏃‍♀️ Spiritus<br />🚩 Ethereum Uncensored<br />👨‍👩‍👧‍👦 Reunited<br />💨 Woosh</p>&mdash; ETHGlobal (@ETHGlobal) <a href="https://twitter.com/ETHGlobal/status/1561502158767415298?ref_src=twsrc%5Etfw">August 21, 2022</a></blockquote>
<script async="" src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>

<h2 id="2022-ethamsterdam">2022 ETHAmsterdam</h2>

<blockquote class="twitter-tweet"><p lang="en" dir="ltr">We couldn&#39;t be more excited to announce our 13 <a href="https://twitter.com/hashtag/ETHAmsterdam?src=hash&amp;ref_src=twsrc%5Etfw">#ETHAmsterdam</a> Finalists!<br /><br />📼 Retr0x<br />💡Nimi.eth<br />⬢ Hexagons Protocol<br />📜 Democrazy<br />🪄 FLOAN<br />🍤 Tempra<br />🤝 pfpnative<br />🚀 Daora<br />📈 Yieldgate<br />🧱 Detris<br />🚴‍♀️ Activitydao<br />👀 Creatorsight<br />✊ Collective Action<br /><br />Thread🧵👇</p>&mdash; ETHGlobal (@ETHGlobal) <a href="https://twitter.com/ETHGlobal/status/1518234859323408384?ref_src=twsrc%5Etfw">April 24, 2022</a></blockquote>
<script async="" src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>

<h2 id="2020-ethlondonuk">2020 ETHLondonUK</h2>

<p><a href="https://ethlondon.devpost.com/project-gallery">https://ethlondon.devpost.com/project-gallery</a></p>

<h2 id="2019-ethwaterloo">2019 ETHWaterloo</h2>

<p><a href="https://ethwaterloo2.devpost.com/project-gallery">https://ethwaterloo2.devpost.com/project-gallery</a></p>

<h2 id="2019-ethboston">2019 ETHBoston</h2>

<p><a href="https://ethboston.devpost.com/project-gallery">https://ethboston.devpost.com/project-gallery</a></p>

<h2 id="2019-ethnewyork">2019 ETHNewYork</h2>

<p><a href="https://ethnewyork.devpost.com/project-gallery">https://ethnewyork.devpost.com/project-gallery</a></p>

<h2 id="2019-ethcapetown">2019 ETHCapeTown</h2>

<p><a href="https://ethcapetown.devpost.com/project-gallery">https://ethcapetown.devpost.com/project-gallery</a></p>

<h2 id="2019-ethparis">2019 ETHParis</h2>

<p><a href="https://ethparis.devpost.com/project-gallery">https://ethparis.devpost.com/project-gallery</a></p>

<h2 id="2018-ethsingapore">2018 ETHSingapore</h2>

<p>44 submissions
<a href="https://ethsingapore.devpost.com/project-gallery">https://ethsingapore.devpost.com/project-gallery</a></p>

<h2 id="2018-ethsanfrancisco">2018 ETHSanFrancisco</h2>

<p>121 submissions
<a href="https://ethsanfrancisco.devpost.com/submissions">https://ethsanfrancisco.devpost.com/submissions</a></p>

<h2 id="2018-ethberlin">2018 ETHBerlin</h2>

<h3 id="winners">winners</h3>

<p><a href="https://medium.com/ethberlin/ethberlin-meet-the-winners-50-70-word-reviews-featuring-ecf-ethprize-gitcoin-and-more-5e98d260b530">https://medium.com/ethberlin/ethberlin-meet-the-winners-50-70-word-reviews-featuring-ecf-ethprize-gitcoin-and-more-5e98d260b530</a></p>

<h3 id="submissions">submissions</h3>

<p>65 submissions
<a href="https://ethberlin.devpost.com/submissions">https://ethberlin.devpost.com/submissions</a></p>

<h2 id="2018-ethindia">2018 ETHIndia</h2>

<h3 id="top-7">top 7</h3>

<p><a href="https://ethindia.devfolio.co/projects?prizes=961223f644744094b25e889e788f187f&amp;show_winners=false">https://ethindia.devfolio.co/projects?prizes=961223f644744094b25e889e788f187f&amp;show_winners=false</a></p>

<h3 id="53-submissions">53 submissions</h3>

<p><a href="https://ethindia.devfolio.co/submissions">https://ethindia.devfolio.co/submissions</a></p>

<h2 id="2018-ethbuenosaires">2018 ETHBuenosAires</h2>

<p>34 submissions
<a href="https://ethbuenosaires.devpost.com/project-gallery">https://ethbuenosaires.devpost.com/project-gallery</a></p>

<h2 id="2018-ethdenver">2018 ETHDenver</h2>

<p>131 submissions
<a href="https://ethdenver.devpost.com/project-gallery">https://ethdenver.devpost.com/project-gallery</a></p>

<h2 id="2017-ethwaterloo">2017 ETHWaterloo</h2>

<p>72 submissions here
<a href="https://ethwaterloo.devpost.com/project-gallery">https://ethwaterloo.devpost.com/project-gallery</a></p>]]></content><author><name>Tomohiko Araki</name></author><summary type="html"><![CDATA[Best of bests in the scene has power to stimulate yourself. I have picked up (not all) repositories of finalists of ETHGlobal. 2023 ETHGlobal New York We are proud to announce:The 13 ETHGlobal New York 🗽 finalists! 🖼 FRAMED!🤑 F.A.S.T.🔍 ZeroTrustBounty🤔 Doom Arena🗺 AirTracker 🔗 SafeCreate2🤖 ConsciousNFT🛸 Space Guardians👁 A.S.R.🥷 MEVictim Rebate📈 RealReturn💰 Abstract Wallet🔒 XSafe👇/🧵&mdash; ETHGlobal (@ETHGlobal) September 24, 2023 FRAMED FRAMED: A fully onchain &amp; trustless hidden information game (Mafia) built with fhEVM Project page https://ethglobal.com/showcase/framed-5vv6q Repository https://github.com/joeyiny/FRAMED Technologies Inco Network fhEVM https://www.inco.network/ Privvy https://www.privy.io/ Homomorphic Encryption F.A.S.T F.A.S.T. is the fastest way to use the blockchain to interact between 2 users. Take a picture of youself to sign up, and then snap a picture with someone else, to then interact with their address seamlessly. Project page https://ethglobal.com/showcase/f-a-s-t-1ian9 Repository https://github.com/SwineCoder101/FAST-ETH-NYC Technologies Telegram Vercel Airstack Safe Mask ZeroTrustBounty We are solving challenges of traditional bug bounty platforms where businesses have to trust 3rd parties with sensitive data and whitehats have no guarantee of payment for their services Project page https://ethglobal.com/showcase/zerotrustbounty-dbs0b Repository https://github.com/antojoseph/ZeroTrustBounty Technologies TLS Notary Lit protocol Doom Arena Distributed DApp for managing and creating contests around the Doom Games. Project page https://ethglobal.com/showcase/doom-arena-hmu4j Repository https://github.com/felipefg/doom-arena Technologies Cartesi AirTracker Decentralized physical asset tracking using AirTags and blockchain storage and messaging Project page https://ethglobal.com/showcase/airtracker-8qstn Repository https://github.com/tkeith/airtracker Technologies AirTag IPFS XMTP Airstack SafeCreate2 Secure multi-chain smart contract deployments using a single-chain Safe Project page https://ethglobal.com/showcase/safecreate2-szzw3 Repository https://github.com/jaydenwindle/safe-create2 Technologies Safe Axelar https://axelar.network/ ConsciousNFT AI-powered tools for NFT projects to awaken conscious identities within your NFTs Project page https://ethglobal.com/showcase/consciousnft-4fbkx Repository https://github.com/louiswilbrink/consciousNFT Technologies GPT-3.5 by OpenAI IPFS Space Guardians Satellite Secured Access Recovery - A MetaMask Guardian management system using Snaps Project page https://ethglobal.com/showcase/space-guardians-bzcfc Repository https://github.com/cryptosat/satlock Technologies Metamask Snaps https://metamask.io/snaps/ Crypto-satellites https://www.cryptosat.io/ A.S.R You’re the owner of a smart contract but you lost the key. You’re screwed…. or are you? Anti Social Recovery is a design pattern that enables users to recover ownership of their smart contract after losing their signing keys. Project page https://ethglobal.com/showcase/a-s-r-spakz Repository https://github.com/crystaltai/anti-social-recovery-club Technologies World ID ThirdWeb https://thirdweb.com/ MEVictim Rebate Designing better market incentives by identifying MEV victims using historical on chain data and airdropping victims a token to access a token gated Uniswap v4 pool on Scroll Project page https://ethglobal.com/showcase/mevictim-rebate-qsxak Repository https://github.com/mliuuu/ETHNYC23/ Technologies AXION https://www.axiom.xyz/ Hyperlane The Graph Uniswap v4 + Hook Launch on Scroll Flashbots RealReturn RealReturn is an open leaderboard of verified real investment returns, enabled by TLSNotary and Jomo. Project page https://ethglobal.com/showcase/realreturn-pu64y Repository https://github.com/chcharcharlie/realreturn Technologies TLSNotary Jomo https://docs.jomo.id/welcome-to-jomo/jomo-overview Abstract Wallet Wallet abstraction enables instant single click onboarding of a user. With nothing but a smartphone using the power of account abstraction with passkeys. generate passkeys credentials, create its smart acount, mint a NFT, all in a single click ! The wallet is invisible. Project page https://ethglobal.com/showcase/abstract-wallet-a9pzf Repository https://github.com/qd-qd/wallet-abstraction Technologies FIDO passkey solidity FCL library (secp256R1 XSafe X-Safe streamlines Gnosis Safe multi-sigs for cross-chain use. Sign once, and the signature auto-propagates to all linked chains. Execute single transactions across multiple networks. Save time, cut overhead, and keep strong security. Project page https://ethglobal.com/showcase/xsafe-bbj01 Repository https://github.com/cucupac/x-safe Technologies Safe 2023 ETHGlobal Paris 🥁 The 13 ETHGlobal Paris 🇫🇷 finalists are:🎙 ZK Microphone📈 GLSwaps🐦 TwitterCampaign💥 TyphoonCashX🪄 Kinetex Light Clients🤫 Harpos🫧 Bubbles🌳 Stake Garden🔐 Cypher Deposit⚡️ Bob the Solver🗣 VoiceSense🛗 Community-bound💬 On-chain messaging aggregator👇/🧵&mdash; ETHGlobal (@ETHGlobal) July 23, 2023 ZK Microphone ZK Microphone: Trusted audio in the age of deepfakes 🔒🎙 Generative AI is a threat to society. It enables disinformation, manipulation, and political subversion. We’ve built the world’s first attested microphone and used ZK-SNARKs to protect authenticity and privacy. Project page https://ethglobal.com/showcase/zk-microphone-8161v Repository https://github.com/Miyamura80/ZKMicrophone Technologies Raspberry Pi Noir https://noir-lang.org/ IPFS GLSwaps Users now have access to spot and perpetual markets from the Uniswap app (app.uniswap.org) using the GLS browser extension routing through GMX as well. From Uniswap, users can access on-chain perpetual markets. Injecting the smart contracts for derivatives by adding a leverage button and slider. For degens. Project page https://ethglobal.com/showcase/glswaps-grandma-leveraged-swaps-gzit8 Repository https://github.com/jaimindp/UniPerp Technologies A browser extension GMX Uniswap dydx TwitterCampaign Twitter Campaigns is an A.I.-enabled on-chain DAO-tool aimed at crypto-familiar users where anyone can create, and participate in on-chain Twitter marketing campaigns. Project page https://ethglobal.com/showcase/twittercampaign-wv9we Repository https://github.com/myounatan/twitter-campaign Technologies Safe AuthKit Biconomy’s Transak SDK TheGraph TyphoonCashX TyphoonCashX is a protocol enabling bridging any ERC20 native tokens across any EVM compatible chain in an anonymous fashion. Project page https://ethglobal.com/showcase/typhooncashx-p10yd Repository https://github.com/TyphoonCashX Technologies Sismo Hyperlane Kinetex Light Clients Kinetex Light Clients for Cross-Chain Resolving (based on ChainLink CCIP, Hyperline, Axelar, ZetaChain &amp; Hashi) Project page https://ethglobal.com/showcase/kinetex-light-clients-mmr-uvzr1 Repository https://github.com/KinetexNetwork/ethglobal-paris Technologies Merkle Mountain Range structure LightClient was inspired by Axiom Hashi adapters Kinetex Protocol 1inch Fusion Harpos Leveraging Sismo and XMTP, Harpos is a social network centered on privacy and anonymity where users can choose multiple criteria to select who they want to connect with. Project page https://ethglobal.com/showcase/harpos-0wpcc Repository https://github.com/ahippoly/EthParisHackathon Technologies Sismo XMTP IPFS Bubbles Bubbles is a voice chat app for onchain communities. Create token-gated rooms based on token ownership, POAP claims, group affiliations and more. Project page https://ethglobal.com/showcase/bubbles-avp6o Repository https://github.com/gskril/ethglobal-paris-hack Technologies Daily.co Airstack Sismo wagmi Stake Garden StakeGarden is a Self Managed Liquid Staking Tokens Portfolio that allows you to diversify and mitigate the risk keeping you safe and in control. Project page https://ethglobal.com/showcase/stake-garden-7tj7j Repository https://github.com/StakeGarden/stakegarden Technologies Metamask SDK 1inch TheGraph Cypher Deposit Elevating financial privacy in crypto withdrawals. Securely withdraw funds via anonymous transfers &amp; secure transactions, preserving anonymity. Project page https://ethglobal.com/showcase/cypher-deposit-cnhs5 Repository https://github.com/Alice-s-Deposit Technologies HOPR https://hoprnet.org/ zkBob IPFS Bob the Solver Infrastructure for Intent-based transactions for better UX of wallets and dapps. The project has successfully developed a ‘solver’ that accepts the user’s intent, automatically formulates the appropriate transactions, and executes them on behalf of the user. Project page https://ethglobal.com/showcase/bob-the-solver-yxtgp Repository https://github.com/nlok5923/solver Technologies Account Abstraction (AA) Wallet a machine learning (ML) model that classifies the user’s intent VoiceSense VoiceSence leverages the power of AI voice recognition to make web3 more inclusive. Project page https://ethglobal.com/showcase/voicesense-yfzpq Repository (link is not found) Technologies Cartesi Hugging Face AI model Community-bound Community Bound Tokens (CBTs) are ERC-20 tokens that can only be transferred among members of a community, which can be used for reputation, community merit based governance, opt-in social credit score, coordination games or engagement scores. Project page https://ethglobal.com/showcase/community-bound-y73ys Repository https://github.com/TroopFinance/community-bound Technologies Safe (forked) Push protocol On-chain messaging aggregator Aggregate all web3 messaging app into S3nd! 🌐 Your one-stop hub for all Web3 messaging apps. Easy single sign-on, Human verification with Worldcoin ✔️ , &amp; native on-chain messaging support. No bots, only genuine convos! Project page https://ethglobal.com/showcase/on-chain-messaging-aggregator-3eshd Repository https://github.com/RafalZgoda/Onchain-Messaging-Aggregator Technologies XMTP Worldcoin Airstack Sismo Message protocols aggregated Native on-chain messaging Debank Hi Blockscan Push protocol 2023 ETHGlobal Waterloo 🥁 We&#39;re proud to announce... The 11 ETHGlobal Waterloo 2023 🍁 finalists! 💦 AquaNet🌌 Smarter Contract🧧 Fukuro🧉 Roll a Mate🐷 Piggybank 6551 NFT😎 Smile DAO🔐 a(i)udit🤖 Tokenbound Titans🛒 zBay⛑ Token Rescue Buddy🎨 Copix👇/🧵&mdash; ETHGlobal (@ETHGlobal) June 25, 2023 AquaNet A social network designed for AI-Driven NFTs to interact with each other and develop unique personalities. Project page https://ethglobal.com/showcase/aquanet-dzz2c Repository https://github.com/AquaNet-ETH-Waterloo-2023/web Technologies Non-fungible Tokenbound Accounts (ERC 6551) OpenAI’s GPT-3.5 Airstack https://docs.tokenbound.org/ Smarter Contract Smarter Contract, powered by Hyperlane, optimizes smart contract performance by intelligently delegating functions to the most cost-efficient blockchain in real-time. It’s a leap towards efficient, accessible, and cost-effective blockchain interaction. Project page https://ethglobal.com/showcase/smarter-contract-kw41h Repository https://github.com/maancham/smarter-contracts Technologies Hyperlane Fukuro Bundle curations with EIP 6551. Auction marketplace for EIP 6551 supported tokens. Project page https://ethglobal.com/showcase/fukuro-3cdwv Repository https://github.com/banky/fukuro Technologies custom implementation of EIP 6551 Pinata wagmi The Graph Roll a Mate Send money on Ethereum Mainnet with transfers costing from zero to 4 cents. By manipulating the mempool to validate and broadcast low gas transactions, which are then included into sidechain orderbook rollups (without user interaction with it) and then likely to be dropped (at zero cost), Ethereum Mainnet users can send and receive payments at zero to 4 cents costs, even faster than the Mainnet transaction completion time. Project page https://ethglobal.com/showcase/roll-a-mate-ob574 Repository https://github.com/ArturVargas/Mate-contracts Technologies The Graph UMA Telegram Lit protocol Airstack XMTP Piggybank 6551 NFT Project page https://ethglobal.com/showcase/piggybank-6551-nft-e2ai5 Repository https://github.com/nnnnicholas/6551piggybank Technologies in progress Smile DAO Project page https://ethglobal.com/showcase/smile-dao-xogfo Repository https://github.com/smiledao/frontend Technologies in progress a(i)udit Project page https://ethglobal.com/showcase/a-i-udit-ja32i Repository https://github.com/leeederek/aiudit/ Technologies in progress Tokenbound Titans Project page https://ethglobal.com/showcase/tokenbound-titans-5w6oq Repository https://github.com/hamzah2304/tokenbound-titans Technologies Risc Zero in progress zBay zBay revolutionizes online marketplaces by decentralizing eBay, empowering merchants and buyers. It enables seamless data and reputation import from eBay, secure escrow, XMTP chat, and transparent dispute resolution using the Uma optimistic oracle Project page https://ethglobal.com/showcase/zbay-r0724 Repository https://github.com/wannabehero/ethglobal-waterloo Technologies Sismo Connect Gitcoin Passport score eBay related ebay profile importer (via puppeteer) Apify IPFS XMTP Airstack UMA TheGraph Token Rescue Buddy Project page https://ethglobal.com/showcase/token-rescue-buddy-bd0sd Repository https://github.com/xavierdmello/TokenRescueBuddy Technologies in progress Copix Project page https://ethglobal.com/showcase/copix-28x5i Repository https://github.com/jaxonL/copix Technologies in progress 2023 ETHGlobal Lisbon We&#39;re proud to announce: our 10 ETHGlobal Lisbon Finalists! 🇵🇹🐶 Credential Corgi😎 Dookies👬 buddy💻 LocalPro🪙 CryptoBureau🤝 Pinky Protocol🤖 Web3-gpt4💸 Donation Station🤯 ChainGPT🗜 op-kompressor 👇 / 🧵&mdash; ETHGlobal (@ETHGlobal) May 14, 2023 Credential Corgi Credential Corgi enables creation of certification standards, issuance of ZKP-compatible certificates, and a request-proof exchange between 3rd parties and credential holders Project page https://ethglobal.com/showcase/credential-corgi-a7a0r Repository https://github.com/tkeith/credential-corgi Technologies SnarkyJS, without MINA GPT-4 Wagmi RainbowKit Dookies Project page https://ethglobal.com/showcase/dookies-0g13u Repository https://github.com/D00ki3s Technologies in progress buddy A Tinder-style app on Lens for developers to find hackathon partners, using Sismo Connect to prove knowledge and skillsof crypto Project page https://ethglobal.com/showcase/buddy-t7gbs Repository https://github.com/catmcgee/buddy Technologies Lens Sismo connect LocalPro Project page https://ethglobal.com/showcase/localpro-xckra Repository https://github.com/wannabehero/ethglobal-lisbon Technologies in progress CryptoBureau CryptoBureau uses the power of Zero Knowledge to build credit score which can allow under-collateralized lending. Project page https://ethglobal.com/showcase/cryptobureau-u1389 Repository https://github.com/EthGlobalLisbon2023/local-pro Technologies Sismo zokrates TrueLayer API PolygonID WorldID Pinky Protocol Project page https://ethglobal.com/showcase/pinky-protocol-vsos2 Repository https://github.com/pinky-io/pinky-protocol Technologies in progress Web3-gpt4 Project page https://ethglobal.com/showcase/web3-gpt4-jkaja Repository https://github.com/Markeljan/web3-gpt4 Technologies in progress Donation Station Project page https://ethglobal.com/showcase/donation-station-g09ok Repository https://github.com/lennardevertz/ethLisbon2023 Technologies in progress ChainGPT Decentralised &amp; verifiable chat AI, backed by the blockchain: a port of Alpaca LLM model leveraging the Cartesi app-specific rollups Project page https://ethglobal.com/showcase/chaingpt-and-accountability-for-ai-agents-doquk Repository https://github.com/dhood/rollups-examples Technologies Porting of Alpaca.cpp into RISC-V Stanford’s Alpaca Large Language Model (LLM) Cartesi op-kompressor Decompress calldata at runtime to save L1 gas while transacting on L2s like Optimism where gas is an expensive resource. Project page https://ethglobal.com/showcase/op-kompressor-1uboo Repository https://github.com/clabby/op-kompressor Technologies in progress 2023 ETHGlobal Tokyo We are proud to announce⎯the 11 ETHGlobal Tokyo finalists! 🇯🇵🤩 POMPoarding🎙️ ZKVoiceKey🔥 BAILOUT➗ AokiApp NFT🐙 Octoplorer🌸 Myna🪧 https://t.co/L5yS0lSEyv💚 AbswapX🦝 Tanuki🤫 YORU🎮 SuperPlay👇 / 🧵&mdash; ETHGlobal (@ETHGlobal) April 16, 2023 2022 ETHIndia We&#39;re proud to announce⎯the 12 ETHIndia 🇮🇳 2022 finalists! ⛑ Web3Rescue🔗 ChainPe🚨 Block Alerts🎳 Bowled💳 DePay🔎 Persona bot-wallet detection⭐️ Sadaiv CI🤝 Proof Of Trust🔒 ETHGuard🔋 ChargeSwap🎉 TRIKL🦺 SafeGuard👇/🧵&mdash; ETHGlobal (@ETHGlobal) December 16, 2022 2022 ETHSanFrancisco We&#39;re excited to announce⎯ the 13 ETHSanFrancisco 2022 finalists!🌎 mundo⚡️ FuelScape📟 BB3 - BEEP3R🏛 DAOCensus🍉 melo vault🔑 Org3💎 UNIC9️⃣ ZK Section 9🛋 Decentral Couch🌐 Cambria🏃‍♂️ Swallow 👻 Peekaboo⛓ Snapchain👇/🧵&mdash; ETHGlobal (@ETHGlobal) November 7, 2022 2022 ETHBogotá We&#39;re proud to announce...The 12 ETHBogota finalists!👇/🧵🧑‍🤝‍🧑 frens🔎 Blobscan Explorer🤑 Bet Streams 🤑👍👎 ETHernal Reviews🍊 PULP🌱🤑 Superseed🌐 Coedia Network🪶 Zipline 🧂Anonymous Vickrey Auctions On Chain🦨 diZKreet 🎧 ClubSpace🧨 txn. xyz&mdash; ETHGlobal (@ETHGlobal) October 9, 2022 2022 ETHMexico We are proud to announce ─ the 12 #ETHMexico finalists! 🇲🇽🌌 Datamorpho🔑Keyppi♦️ Kiwik🛍 Sell Out🏆 Emergence📚 Staxx🏘 BlockCommunity🤑 ETHPoint🏃‍♀️ Spiritus🚩 Ethereum Uncensored👨‍👩‍👧‍👦 Reunited💨 Woosh&mdash; ETHGlobal (@ETHGlobal) August 21, 2022 2022 ETHAmsterdam We couldn&#39;t be more excited to announce our 13 #ETHAmsterdam Finalists!📼 Retr0x💡Nimi.eth⬢ Hexagons Protocol📜 Democrazy🪄 FLOAN🍤 Tempra🤝 pfpnative🚀 Daora📈 Yieldgate🧱 Detris🚴‍♀️ Activitydao👀 Creatorsight✊ Collective ActionThread🧵👇&mdash; ETHGlobal (@ETHGlobal) April 24, 2022 2020 ETHLondonUK https://ethlondon.devpost.com/project-gallery 2019 ETHWaterloo https://ethwaterloo2.devpost.com/project-gallery 2019 ETHBoston https://ethboston.devpost.com/project-gallery 2019 ETHNewYork https://ethnewyork.devpost.com/project-gallery 2019 ETHCapeTown https://ethcapetown.devpost.com/project-gallery 2019 ETHParis https://ethparis.devpost.com/project-gallery 2018 ETHSingapore 44 submissions https://ethsingapore.devpost.com/project-gallery 2018 ETHSanFrancisco 121 submissions https://ethsanfrancisco.devpost.com/submissions 2018 ETHBerlin winners https://medium.com/ethberlin/ethberlin-meet-the-winners-50-70-word-reviews-featuring-ecf-ethprize-gitcoin-and-more-5e98d260b530 submissions 65 submissions https://ethberlin.devpost.com/submissions 2018 ETHIndia top 7 https://ethindia.devfolio.co/projects?prizes=961223f644744094b25e889e788f187f&amp;show_winners=false 53 submissions https://ethindia.devfolio.co/submissions 2018 ETHBuenosAires 34 submissions https://ethbuenosaires.devpost.com/project-gallery 2018 ETHDenver 131 submissions https://ethdenver.devpost.com/project-gallery 2017 ETHWaterloo 72 submissions here https://ethwaterloo.devpost.com/project-gallery]]></summary></entry><entry><title type="html">You can prepare Server Side Include env in 202x</title><link href="https://metaproxy.co/SSI/" rel="alternate" type="text/html" title="You can prepare Server Side Include env in 202x" /><published>2023-04-30T00:00:00+00:00</published><updated>2023-04-30T00:00:00+00:00</updated><id>https://metaproxy.co/SSI</id><content type="html" xml:base="https://metaproxy.co/SSI/"><![CDATA[<p>I have never imagined I met again this arcane legacy forgery in 202x, but it’s real. From my old memories, I have been prohibited to enable SSI in Apache or nginx from my ethos, higher-self or something that governs me. SSI is very easy to use, but in old days we have faced tons of exploits and this is why SSI is not popular these days.
<a href="https://crashtest-security.com/server-side-includes/">https://crashtest-security.com/server-side-includes/</a></p>

<p>Anyway this is cheatsheet when you have met SSI code and need to prepare development environment for maintain or fix something in that code.</p>

<h2 id="how-to-do-it">How to do it</h2>

<h3 id="install-node">Install node</h3>

<p>Install node.</p>

<h3 id="prepare-modules">Prepare modules</h3>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>npm <span class="nb">install </span>connect-ssi express
</code></pre></div></div>

<h3 id="prepare-appjs">Prepare app.js</h3>

<p>Place app.js in the directory of other contents resides.</p>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">const</span> <span class="nx">express</span>         <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">express</span><span class="dl">'</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">path</span>            <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">path</span><span class="dl">'</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">connectSSI</span>      <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">connect-ssi</span><span class="dl">'</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">app</span>             <span class="o">=</span> <span class="nx">express</span><span class="p">();</span>
<span class="kd">const</span> <span class="nx">port</span>            <span class="o">=</span> <span class="mi">3000</span><span class="p">;</span>
<span class="kd">const</span> <span class="nx">ssiDirectory</span>    <span class="o">=</span> <span class="nx">path</span><span class="p">.</span><span class="nx">join</span><span class="p">(</span><span class="nx">__dirname</span><span class="p">,</span> <span class="dl">'</span><span class="s1">./</span><span class="dl">'</span><span class="p">);</span> 
<span class="kd">const</span> <span class="nx">extSSIvalidated</span> <span class="o">=</span> <span class="dl">'</span><span class="s1">.html</span><span class="dl">'</span> <span class="c1">// The file extension for your SSI files</span>


<span class="kd">const</span> <span class="nx">ssiMiddleware</span> <span class="o">=</span> <span class="nx">connectSSI</span><span class="p">({</span>
  <span class="na">baseDir</span><span class="p">:</span> <span class="nx">ssiDirectory</span><span class="p">,</span>
  <span class="na">ext</span>    <span class="p">:</span> <span class="nx">extSSIvalidated</span>
<span class="p">});</span>
<span class="nx">app</span><span class="p">.</span><span class="nx">use</span><span class="p">(</span><span class="nx">ssiMiddleware</span><span class="p">);</span>
<span class="nx">app</span><span class="p">.</span><span class="nx">use</span><span class="p">(</span><span class="nx">express</span><span class="p">.</span><span class="kd">static</span><span class="p">(</span><span class="nx">ssiDirectory</span><span class="p">));</span>

<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span>  <span class="s2">`Content is located at </span><span class="p">${</span><span class="nx">ssiDirectory</span><span class="p">}</span><span class="s2">`</span><span class="p">);</span>
<span class="nx">app</span><span class="p">.</span><span class="nx">listen</span><span class="p">(</span><span class="nx">port</span><span class="p">,</span> <span class="dl">'</span><span class="s1">0.0.0.0</span><span class="dl">'</span><span class="p">,</span> <span class="p">()</span> <span class="o">=&gt;</span> <span class="p">{</span>
  <span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s2">`Server is running  at http://localhost:</span><span class="p">${</span><span class="nx">port</span><span class="p">}</span><span class="s2">`</span><span class="p">);</span>
<span class="p">});</span>
</code></pre></div></div>

<h3 id="run-it">Run it</h3>

<p>Run and access to http://localhost:3000.</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>node app.js
</code></pre></div></div>]]></content><author><name>Tomohiko Araki</name></author><summary type="html"><![CDATA[I have never imagined I met again this arcane legacy forgery in 202x, but it’s real. From my old memories, I have been prohibited to enable SSI in Apache or nginx from my ethos, higher-self or something that governs me. SSI is very easy to use, but in old days we have faced tons of exploits and this is why SSI is not popular these days. https://crashtest-security.com/server-side-includes/ Anyway this is cheatsheet when you have met SSI code and need to prepare development environment for maintain or fix something in that code. How to do it Install node Install node. Prepare modules npm install connect-ssi express Prepare app.js Place app.js in the directory of other contents resides. const express = require('express'); const path = require('path'); const connectSSI = require('connect-ssi'); const app = express(); const port = 3000; const ssiDirectory = path.join(__dirname, './'); const extSSIvalidated = '.html' // The file extension for your SSI files const ssiMiddleware = connectSSI({ baseDir: ssiDirectory, ext : extSSIvalidated }); app.use(ssiMiddleware); app.use(express.static(ssiDirectory)); console.log( `Content is located at ${ssiDirectory}`); app.listen(port, '0.0.0.0', () =&gt; { console.log(`Server is running at http://localhost:${port}`); }); Run it Run and access to http://localhost:3000. node app.js]]></summary></entry><entry><title type="html">LINE bot for generating pixelart.</title><link href="https://metaproxy.co/LINEBot/" rel="alternate" type="text/html" title="LINE bot for generating pixelart." /><published>2023-02-17T00:00:00+00:00</published><updated>2023-02-17T00:00:00+00:00</updated><id>https://metaproxy.co/LINEBot</id><content type="html" xml:base="https://metaproxy.co/LINEBot/"><![CDATA[<p>A LINE bot for generating pixel art, coded by python and running in CloudFunctions.</p>

<h2 id="how-it-works">How it works</h2>

<p>This time, I have used google cloud functions as Webhook endpoint that receives request from Users through LINE channel, and used openAI’s DALL-E for generating pixel art pics.</p>

<pre><code class="language-mermaid">graph LR;
  A[User];
  B[LineBot];
  C[CloudFunctions];
  D[OpenAI];

  A --&gt; B --&gt; C --&gt; D

</code></pre>

<h2 id="implementation">Implementation</h2>

<h2 id="prepare-image-generator">Prepare Image Generator</h2>

<h3 id="grab-openais-api-key">Grab OpenAI’s API KEY</h3>

<p>This is best. Just follow guide and be getting started.
Alongside your 1st try, you will get the API KEY accordingly.
<a href="https://platform.openai.com/docs/quickstart">https://platform.openai.com/docs/quickstart</a></p>

<h3 id="prepare-webhook-endpoint-in-cloudfunctions">Prepare webhook endpoint in CloudFunctions</h3>

<p>Sorry I am not good teacher that can write guide step by step, but I believe the best one to explain how to be get started in Cloud function is Mr/Ms Google right?
<a href="https://cloud.google.com/functions/docs/create-deploy-http-python">https://cloud.google.com/functions/docs/create-deploy-http-python</a></p>

<p>Alongside your 1st try, you will get the appropriate env, but I just would like to add some tips.</p>

<ul>
  <li>The endpoint of cloud function needs to be anonymous to be called from LINEBot, and you also need to add permission of AllUsers to CloudRun behind cloudFunctions you have created.</li>
  <li>If you are planning to use AudioMessage in LINE, it’s better to choose pyhon3.9 as LINE utilizes AAC(m4a) format as Audio so pydub and ffmpeg are required but ffmpeg do not run in python3.11 and python 3.10 env.</li>
  <li>You need to add proper permissions to the account that runs CloudFunctions, it will be different account for your local account, so basically you do not need GOOGLEAPIKEY variables when code is run in CloudFunctions.</li>
</ul>

<h3 id="deploy-code-to-cloudfunctions">Deploy code to CloudFunctions</h3>

<h4 id="environment-variables">Environment VARIABLES</h4>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">LINE_CHANNEL_ACCESS_TOKEN</span><span class="o">=</span>YOUR_KEY_INFO1
<span class="nv">LINE_CHANNEL_SECRET</span><span class="o">=</span>YOUR_KEY_INFO2
<span class="nv">OPENAI_API_KEY</span><span class="o">=</span>YOUR_KEY_INFO3
</code></pre></div></div>

<h4 id="mainpy">main.py</h4>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">functions_framework</span>
<span class="kn">import</span> <span class="nn">requests</span>
<span class="kn">import</span> <span class="nn">json</span>
<span class="kn">import</span> <span class="nn">os</span>
<span class="kn">import</span> <span class="nn">html</span>
<span class="kn">import</span> <span class="nn">re</span>
<span class="kn">from</span> <span class="nn">dotenv</span> <span class="kn">import</span> <span class="n">load_dotenv</span><span class="p">,</span> <span class="n">find_dotenv</span>
<span class="kn">from</span> <span class="nn">linebot</span> <span class="kn">import</span> <span class="n">LineBotApi</span><span class="p">,</span> <span class="n">WebhookHandler</span>
<span class="kn">from</span> <span class="nn">linebot.exceptions</span> <span class="kn">import</span> <span class="n">InvalidSignatureError</span>
<span class="kn">from</span> <span class="nn">linebot.models</span> <span class="kn">import</span> <span class="n">MessageEvent</span><span class="p">,</span> <span class="n">TextMessage</span><span class="p">,</span> <span class="n">TextSendMessage</span><span class="p">,</span> <span class="n">ImageSendMessage</span><span class="p">,</span> <span class="n">VideoSendMessage</span><span class="p">,</span> <span class="n">TemplateSendMessage</span><span class="p">,</span> <span class="n">CarouselTemplate</span><span class="p">,</span> <span class="n">CarouselColumn</span>

<span class="n">load_dotenv</span><span class="p">(</span><span class="n">find_dotenv</span><span class="p">())</span>
<span class="n">line_bot_api</span> <span class="o">=</span> <span class="n">LineBotApi</span><span class="p">(</span><span class="n">os</span><span class="p">.</span><span class="n">getenv</span><span class="p">(</span><span class="s">"LINE_CHANNEL_ACCESS_TOKEN"</span><span class="p">))</span>
<span class="n">handler</span>      <span class="o">=</span> <span class="n">WebhookHandler</span><span class="p">(</span><span class="n">os</span><span class="p">.</span><span class="n">getenv</span><span class="p">(</span><span class="s">"LINE_CHANNEL_SECRET"</span><span class="p">))</span>

<span class="n">IMAGE_QUERY_URL</span>    <span class="o">=</span> <span class="s">"https://api.openai.com/v1/images/generations"</span>
<span class="n">openai_key</span>   <span class="o">=</span> <span class="n">os</span><span class="p">.</span><span class="n">getenv</span><span class="p">(</span><span class="s">"OPENAI_API_KEY"</span><span class="p">)</span>

<span class="o">@</span><span class="n">functions_framework</span><span class="p">.</span><span class="n">http</span>
<span class="k">def</span> <span class="nf">hello_http</span><span class="p">(</span><span class="n">request</span><span class="p">):</span>
   <span class="c1">#return generate_dalle_image_from_request(request) # for invoking from curl
</span>   <span class="k">return</span> <span class="n">lineEventHandler</span><span class="p">(</span><span class="n">request</span><span class="p">)</span>                   <span class="c1"># for invoking from LINE
</span>
<span class="k">def</span> <span class="nf">lineEventHandler</span><span class="p">(</span><span class="n">request</span><span class="p">):</span>
  <span class="n">signature</span> <span class="o">=</span> <span class="n">request</span><span class="p">.</span><span class="n">headers</span><span class="p">[</span><span class="s">'X-Line-Signature'</span><span class="p">]</span>

  <span class="n">body</span> <span class="o">=</span> <span class="n">request</span><span class="p">.</span><span class="n">get_data</span><span class="p">(</span><span class="n">as_text</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
  <span class="k">print</span><span class="p">(</span><span class="s">"Request body: "</span> <span class="o">+</span> <span class="n">body</span><span class="p">)</span>
  <span class="k">try</span><span class="p">:</span>
      <span class="n">handler</span><span class="p">.</span><span class="n">handle</span><span class="p">(</span><span class="n">body</span><span class="p">,</span> <span class="n">signature</span><span class="p">)</span>
  <span class="k">except</span> <span class="n">InvalidSignatureError</span><span class="p">:</span>
      <span class="k">return</span> <span class="s">""</span><span class="p">,</span> <span class="mi">400</span>

  <span class="k">return</span> <span class="s">'OK'</span>

<span class="o">@</span><span class="n">handler</span><span class="p">.</span><span class="n">add</span><span class="p">(</span><span class="n">MessageEvent</span><span class="p">,</span> <span class="n">message</span><span class="o">=</span><span class="n">TextMessage</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">handle_message</span><span class="p">(</span><span class="n">event</span><span class="p">):</span>
    <span class="n">text_message</span>  <span class="o">=</span> <span class="n">TextSendMessage</span><span class="p">(</span><span class="n">text</span><span class="o">=</span><span class="s">"🖍️描いています..."</span><span class="p">)</span>
    <span class="n">url</span> <span class="o">=</span> <span class="n">generate_dalle_image_from_str</span><span class="p">(</span><span class="n">event</span><span class="p">.</span><span class="n">message</span><span class="p">.</span><span class="n">text</span><span class="p">)</span>
    <span class="n">image_message</span> <span class="o">=</span> <span class="n">ImageSendMessage</span><span class="p">(</span><span class="n">original_content_url</span><span class="o">=</span><span class="n">url</span><span class="p">,</span> <span class="n">preview_image_url</span><span class="o">=</span><span class="n">url</span><span class="p">)</span>

    <span class="n">messages</span> <span class="o">=</span> <span class="p">[</span><span class="n">text_message</span><span class="p">,</span> <span class="n">image_message</span><span class="p">]</span>
    <span class="n">line_bot_api</span><span class="p">.</span><span class="n">reply_message</span><span class="p">(</span>
        <span class="n">event</span><span class="p">.</span><span class="n">reply_token</span><span class="p">,</span>
        <span class="n">messages</span><span class="p">)</span>

<span class="k">def</span> <span class="nf">sanitize</span><span class="p">(</span><span class="n">string</span><span class="p">):</span>
    <span class="c1"># Remove HTML tags
</span>    <span class="n">string</span> <span class="o">=</span> <span class="n">html</span><span class="p">.</span><span class="n">escape</span><span class="p">(</span><span class="n">string</span><span class="p">)</span>
    <span class="c1"># Remove potentially harmful characters
</span>    <span class="n">string</span> <span class="o">=</span> <span class="n">re</span><span class="p">.</span><span class="n">sub</span><span class="p">(</span><span class="sa">r</span><span class="s">'[^\w\s]+'</span><span class="p">,</span> <span class="s">''</span><span class="p">,</span> <span class="n">string</span><span class="p">)</span>
    <span class="c1"># Remove JavaScript code
</span>    <span class="n">string</span> <span class="o">=</span> <span class="n">re</span><span class="p">.</span><span class="n">sub</span><span class="p">(</span><span class="sa">r</span><span class="s">'&lt;script.*?&gt;.*?&lt;/script&gt;'</span><span class="p">,</span> <span class="s">''</span><span class="p">,</span> <span class="n">string</span><span class="p">,</span> <span class="n">flags</span><span class="o">=</span><span class="n">re</span><span class="p">.</span><span class="n">DOTALL</span> <span class="o">|</span> <span class="n">re</span><span class="p">.</span><span class="n">IGNORECASE</span><span class="p">)</span>
    <span class="c1"># Remove SQL commands
</span>    <span class="n">string</span> <span class="o">=</span> <span class="n">re</span><span class="p">.</span><span class="n">sub</span><span class="p">(</span><span class="sa">r</span><span class="s">'(\b(ALTER|CREATE|DELETE|DROP|EXEC(UTE){0,1}|INSERT( +INTO){0,1}|MERGE|SELECT|UPDATE|UNION( +ALL){0,1})\b)'</span><span class="p">,</span> <span class="s">''</span><span class="p">,</span> <span class="n">string</span><span class="p">,</span> <span class="n">flags</span><span class="o">=</span><span class="n">re</span><span class="p">.</span><span class="n">IGNORECASE</span><span class="p">)</span>
    <span class="c1"># Remove other possible harmful characters
</span>    <span class="k">return</span> <span class="n">string</span><span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"@"</span><span class="p">,</span> <span class="s">"at"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"."</span><span class="p">,</span> <span class="s">"dot"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"/"</span><span class="p">,</span> <span class="s">"slash"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"?"</span><span class="p">,</span> <span class="s">"question"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"="</span><span class="p">,</span> <span class="s">"equals"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"&amp;"</span><span class="p">,</span> <span class="s">"and"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"%"</span><span class="p">,</span> <span class="s">"percent"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"$"</span><span class="p">,</span> <span class="s">"dollar"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"#"</span><span class="p">,</span> <span class="s">"hash"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"!"</span><span class="p">,</span> <span class="s">"exclamation"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"&gt;"</span><span class="p">,</span> <span class="s">"greater"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"&lt;"</span><span class="p">,</span> <span class="s">"lesser"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">":"</span><span class="p">,</span> <span class="s">"colon"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">";"</span><span class="p">,</span> <span class="s">"semicolon"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"{"</span><span class="p">,</span> <span class="s">"braceleft"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"}"</span><span class="p">,</span> <span class="s">"braceright"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"["</span><span class="p">,</span> <span class="s">"bracketleft"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"]"</span><span class="p">,</span> <span class="s">"bracketright"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"("</span><span class="p">,</span> <span class="s">"parenleft"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">")"</span><span class="p">,</span> <span class="s">"parenright"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"'"</span><span class="p">,</span> <span class="s">"apostrophe"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">'"'</span><span class="p">,</span> <span class="s">"quotemark"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">","</span><span class="p">,</span> <span class="s">"comma"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"~"</span><span class="p">,</span> <span class="s">"tilde"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"`"</span><span class="p">,</span> <span class="s">"backtick"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"^"</span><span class="p">,</span> <span class="s">"caret"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"|"</span><span class="p">,</span> <span class="s">"verticalbar"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"</span><span class="se">\\</span><span class="s">"</span><span class="p">,</span> <span class="s">"backslash"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">" "</span><span class="p">,</span> <span class="s">"-"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"*"</span><span class="p">,</span> <span class="s">"asterisk"</span><span class="p">)</span>

<span class="k">def</span> <span class="nf">generate_prompt</span><span class="p">(</span><span class="n">seed</span><span class="p">):</span>
    <span class="n">seed</span> <span class="o">=</span> <span class="n">sanitize</span><span class="p">(</span><span class="n">seed</span><span class="p">)</span>
    <span class="k">return</span> <span class="s">"""{} as cute pixel art, like octopath traveller. Full color. High resolutions. Nintendo Switch. Kawaii. The background is something fantastic with wander."""</span><span class="p">.</span><span class="nb">format</span><span class="p">(</span>
        <span class="n">seed</span><span class="p">.</span><span class="n">capitalize</span><span class="p">()</span>
    <span class="p">)</span>

<span class="k">def</span> <span class="nf">get_dalle_image</span><span class="p">(</span><span class="n">prompt</span><span class="p">):</span>
    <span class="n">prompt</span> <span class="o">=</span> <span class="n">sanitize</span><span class="p">(</span><span class="n">prompt</span><span class="p">)</span>

    <span class="c1"># Request header
</span>    <span class="n">headers</span> <span class="o">=</span> <span class="p">{</span>
        <span class="s">"Content-Type"</span><span class="p">:</span> <span class="s">"application/json"</span><span class="p">,</span>
        <span class="s">"Authorization"</span><span class="p">:</span> <span class="s">"Bearer {}"</span><span class="p">.</span><span class="nb">format</span><span class="p">(</span><span class="n">openai_key</span><span class="p">)</span>
    <span class="p">}</span>

    <span class="c1"># Request data
</span>    <span class="n">data</span> <span class="o">=</span> <span class="p">{</span>
        <span class="s">"model"</span><span class="p">:</span> <span class="s">"image-alpha-001"</span><span class="p">,</span>
        <span class="s">"prompt"</span><span class="p">:</span> <span class="n">prompt</span><span class="p">,</span>
        <span class="s">"num_images"</span><span class="p">:</span><span class="mi">1</span><span class="p">,</span>
        <span class="s">"size"</span><span class="p">:</span><span class="s">"512x512"</span><span class="p">,</span>
        <span class="s">"response_format"</span><span class="p">:</span><span class="s">"url"</span>
    <span class="p">}</span>

    <span class="c1"># Make a request to DALL-E API
</span>    <span class="n">resp</span> <span class="o">=</span> <span class="n">requests</span><span class="p">.</span><span class="n">post</span><span class="p">(</span><span class="n">IMAGE_QUERY_URL</span><span class="p">,</span> <span class="n">headers</span><span class="o">=</span><span class="n">headers</span><span class="p">,</span> <span class="n">data</span><span class="o">=</span><span class="n">json</span><span class="p">.</span><span class="n">dumps</span><span class="p">(</span><span class="n">data</span><span class="p">))</span>

    <span class="c1"># Check if request is successful
</span>    <span class="k">if</span> <span class="n">resp</span><span class="p">.</span><span class="n">status_code</span> <span class="o">!=</span> <span class="mi">200</span><span class="p">:</span>
        <span class="k">return</span> <span class="s">"Failed to generate image. Error: {}"</span><span class="p">.</span><span class="nb">format</span><span class="p">(</span><span class="n">resp</span><span class="p">.</span><span class="n">content</span><span class="p">)</span>

    <span class="c1"># Return the URL of the generated image
</span>    <span class="n">response_json</span> <span class="o">=</span> <span class="n">resp</span><span class="p">.</span><span class="n">json</span><span class="p">()</span>
    <span class="k">return</span> <span class="n">response_json</span><span class="p">[</span><span class="s">'data'</span><span class="p">][</span><span class="mi">0</span><span class="p">][</span><span class="s">'url'</span><span class="p">]</span>

<span class="k">def</span> <span class="nf">generate_dalle_image_from_str</span><span class="p">(</span><span class="n">s</span><span class="p">):</span>
    <span class="k">return</span> <span class="n">get_dalle_image</span><span class="p">(</span><span class="n">generate_prompt</span><span class="p">(</span><span class="n">s</span><span class="p">))</span>

<span class="k">def</span> <span class="nf">generate_dalle_image_from_request</span><span class="p">(</span><span class="n">request</span><span class="p">):</span>
    <span class="k">if</span> <span class="n">request</span><span class="p">.</span><span class="n">method</span> <span class="o">!=</span> <span class="s">"POST"</span><span class="p">:</span>
        <span class="k">return</span> <span class="s">"Only POST requests are accepted"</span><span class="p">,</span> <span class="mi">405</span>
    <span class="k">elif</span> <span class="ow">not</span> <span class="n">request</span><span class="p">.</span><span class="n">headers</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="s">"Content-Type"</span><span class="p">)</span> <span class="o">==</span> <span class="s">"application/json"</span><span class="p">:</span>
        <span class="k">return</span> <span class="s">"Only application/json requests are accepted"</span><span class="p">,</span> <span class="mi">415</span>
    
    <span class="n">request_json</span> <span class="o">=</span> <span class="n">request</span><span class="p">.</span><span class="n">get_json</span><span class="p">()</span>
    <span class="n">s</span> <span class="o">=</span> <span class="n">request_json</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="s">"prompt"</span><span class="p">)</span>
    <span class="k">return</span> <span class="n">get_dalle_image</span><span class="p">(</span><span class="n">generate_prompt</span><span class="p">(</span><span class="n">s</span><span class="p">))</span>
</code></pre></div></div>

<h4 id="requirementstxt">requirements.txt</h4>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">functions</span><span class="o">-</span><span class="n">framework</span><span class="o">==</span><span class="mf">3.</span><span class="o">*</span>
<span class="n">python</span><span class="o">-</span><span class="n">dotenv</span><span class="o">==</span><span class="mf">0.19</span><span class="p">.</span><span class="mi">2</span>
<span class="n">future</span><span class="o">==</span><span class="mf">0.18</span><span class="p">.</span><span class="mi">3</span>
<span class="n">line</span><span class="o">-</span><span class="n">bot</span><span class="o">-</span><span class="n">sdk</span><span class="o">==</span><span class="mf">2.4</span><span class="p">.</span><span class="mi">1</span>
<span class="n">requests</span><span class="o">==</span><span class="mf">2.28</span><span class="p">.</span><span class="mi">1</span>
<span class="n">openai</span><span class="o">==</span><span class="mf">0.26</span><span class="p">.</span><span class="mi">0</span>
</code></pre></div></div>

<h3 id="make-linebot-as-channel-and-set-cloudfunction-as-webhook-endpoint">Make LineBot as channel and set CloudFunction as Webhook endpoint</h3>

<p>I am very happy to live in this internet era. You can get everyting you need just build small app guided by this:
<a href="https://developers.line.biz/en/docs/messaging-api/building-bot/">https://developers.line.biz/en/docs/messaging-api/building-bot/</a></p>

<h2 id="tips-on-cloud-functions">Tips on Cloud Functions</h2>

<p>You are unable to use pydub ( ffmpeg ) at Python3.11 and 3.10.
Use 3.9 otherwise you will face errors.</p>

<h3 id="rerefence">rerefence</h3>

<p>Cloud function do supports pydub ( ffmpeg ), but Python311 and 310 env are unable to include them as base image itself do not include ffmpeg.
<a href="https://issuetracker.google.com/issues/242685280?pli=1">https://issuetracker.google.com/issues/242685280?pli=1</a></p>

<h3 id="errors">errors</h3>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>File <span class="s2">"/layers/google.python.runtime/python/lib/python3.10/subprocess.py"</span>, line 1847, <span class="k">in </span>_execute_child raise child_exception_type<span class="o">(</span>errno_num, err_msg, err_filename<span class="o">)</span> FileNotFoundError: <span class="o">[</span>Errno 2] No such file or directory: <span class="s1">'ffprobe'</span>
</code></pre></div></div>

<h3 id="appendix">Appendix</h3>

<p>You can build ChatGPT like chatBot using this framework.
I will drop sample code that demonstrates text and voice based communication with GPT in openai’s cloud. Please feel free to contact me if you are curious to implement yours.</p>

<p><a href="https://twitter.com/rtree">https://twitter.com/rtree</a></p>

<h4 id="mainpy-1">main.py</h4>

<h4 id="environment-variables-1">Environment VARIABLES</h4>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">LINE_CHANNEL_ACCESS_TOKEN</span><span class="o">=</span>YOUR_KEY_INFO1
<span class="nv">LINE_CHANNEL_SECRET</span><span class="o">=</span>YOUR_KEY_INFO2
<span class="nv">OPENAI_API_KEY</span><span class="o">=</span>YOUR_KEY_INFO3
<span class="nv">BUCKET_NAME</span><span class="o">=</span>YOUR_BUCKET_NAME
</code></pre></div></div>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">openai</span>
<span class="kn">import</span> <span class="nn">io</span>
<span class="kn">import</span> <span class="nn">os</span>
<span class="kn">import</span> <span class="nn">datetime</span>
<span class="kn">from</span> <span class="nn">dotenv</span> <span class="kn">import</span> <span class="n">load_dotenv</span><span class="p">,</span> <span class="n">find_dotenv</span>
<span class="kn">import</span> <span class="nn">uuid</span>
<span class="kn">import</span> <span class="nn">wave</span>
<span class="kn">from</span> <span class="nn">pydub</span> <span class="kn">import</span> <span class="n">AudioSegment</span>
<span class="kn">import</span> <span class="nn">html</span>
<span class="kn">import</span> <span class="nn">re</span>
<span class="kn">import</span> <span class="nn">random</span>
<span class="kn">from</span> <span class="nn">google.cloud</span> <span class="kn">import</span> <span class="n">texttospeech</span>
<span class="kn">from</span> <span class="nn">google.cloud</span> <span class="kn">import</span> <span class="n">storage</span>
<span class="kn">from</span> <span class="nn">google.cloud</span> <span class="kn">import</span> <span class="n">speech_v1</span> <span class="k">as</span> <span class="n">speech</span>

<span class="c1"># for remote implementation
</span><span class="kn">import</span> <span class="nn">functions_framework</span>
<span class="kn">import</span> <span class="nn">requests</span>
<span class="kn">import</span> <span class="nn">json</span>
<span class="kn">from</span> <span class="nn">linebot</span> <span class="kn">import</span> <span class="n">LineBotApi</span><span class="p">,</span> <span class="n">WebhookHandler</span>
<span class="kn">from</span> <span class="nn">linebot.exceptions</span> <span class="kn">import</span> <span class="n">InvalidSignatureError</span>
<span class="kn">from</span> <span class="nn">linebot.models</span> <span class="kn">import</span> <span class="n">MessageEvent</span><span class="p">,</span> <span class="n">TextMessage</span><span class="p">,</span> <span class="n">TextSendMessage</span><span class="p">,</span> <span class="n">ImageSendMessage</span><span class="p">,</span> <span class="n">VideoSendMessage</span><span class="p">,</span> <span class="n">TemplateSendMessage</span><span class="p">,</span> <span class="n">CarouselTemplate</span><span class="p">,</span> <span class="n">CarouselColumn</span><span class="p">,</span> <span class="n">AudioSendMessage</span><span class="p">,</span> <span class="n">AudioMessage</span>

<span class="n">load_dotenv</span><span class="p">(</span><span class="n">find_dotenv</span><span class="p">())</span>
<span class="n">BUCKET_NAME</span>  <span class="o">=</span> <span class="n">os</span><span class="p">.</span><span class="n">environ</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="s">"BUCKET_NAME"</span><span class="p">)</span>
<span class="n">line_bot_api</span> <span class="o">=</span> <span class="n">LineBotApi</span><span class="p">(</span><span class="n">os</span><span class="p">.</span><span class="n">getenv</span><span class="p">(</span><span class="s">"LINE_CHANNEL_ACCESS_TOKEN"</span><span class="p">))</span>
<span class="n">handler</span>      <span class="o">=</span> <span class="n">WebhookHandler</span><span class="p">(</span><span class="n">os</span><span class="p">.</span><span class="n">getenv</span><span class="p">(</span><span class="s">"LINE_CHANNEL_SECRET"</span><span class="p">))</span>
<span class="n">openai</span><span class="p">.</span><span class="n">api_key</span> <span class="o">=</span> <span class="n">os</span><span class="p">.</span><span class="n">environ</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="s">"OPENAI_API_KEY"</span><span class="p">)</span>

<span class="c1">#def BUCKET_NAME:
#    load_dotenv(find_dotenv())
#    return os.environ.get("BUCKET_NAME")
</span>
<span class="o">@</span><span class="n">functions_framework</span><span class="p">.</span><span class="n">http</span>
<span class="k">def</span> <span class="nf">hello_http</span><span class="p">(</span><span class="n">request</span><span class="p">):</span>
   <span class="c1">#return generate_dalle_image_from_request(request) # for invoking from curl
</span>   <span class="k">return</span> <span class="n">lineEventHandler</span><span class="p">(</span><span class="n">request</span><span class="p">)</span>                   <span class="c1"># for invoking from LINE
</span>
<span class="k">def</span> <span class="nf">lineEventHandler</span><span class="p">(</span><span class="n">request</span><span class="p">):</span>
  <span class="n">signature</span> <span class="o">=</span> <span class="n">request</span><span class="p">.</span><span class="n">headers</span><span class="p">[</span><span class="s">'X-Line-Signature'</span><span class="p">]</span>
  <span class="n">body</span> <span class="o">=</span> <span class="n">request</span><span class="p">.</span><span class="n">get_data</span><span class="p">(</span><span class="n">as_text</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
  <span class="k">print</span><span class="p">(</span><span class="s">"Request body: "</span> <span class="o">+</span> <span class="n">body</span><span class="p">)</span>
  <span class="k">try</span><span class="p">:</span>
      <span class="n">handler</span><span class="p">.</span><span class="n">handle</span><span class="p">(</span><span class="n">body</span><span class="p">,</span> <span class="n">signature</span><span class="p">)</span>
  <span class="k">except</span> <span class="n">InvalidSignatureError</span><span class="p">:</span>
      <span class="k">return</span> <span class="s">""</span><span class="p">,</span> <span class="mi">400</span>

  <span class="k">return</span> <span class="s">'OK'</span>

<span class="o">@</span><span class="n">handler</span><span class="p">.</span><span class="n">add</span><span class="p">(</span><span class="n">MessageEvent</span><span class="p">,</span> <span class="n">message</span><span class="o">=</span><span class="n">TextMessage</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">handle_text_message</span><span class="p">(</span><span class="n">event</span><span class="p">):</span>
    <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">event</span><span class="p">.</span><span class="n">message</span><span class="p">,</span> <span class="n">TextMessage</span><span class="p">):</span>
        <span class="n">text_message</span>  <span class="o">=</span> <span class="n">TextSendMessage</span><span class="p">(</span><span class="n">text</span><span class="o">=</span><span class="s">"🤖考えています..."</span><span class="p">)</span>

        <span class="c1">## reply with the same text as the user sent in voice                
</span>        <span class="c1">#url = synthetize_text_and_upload_public(event.message.text)
</span>        <span class="c1">#duration = str(int(float(url.split("_")[2])*1000))    # in milliseconds
</span>        <span class="c1">#audio_message = AudioSendMessage(original_content_url=url, duration=duration)
</span>
        <span class="n">reply</span> <span class="o">=</span> <span class="n">get_answer_from_openai_test</span><span class="p">(</span><span class="n">event</span><span class="p">.</span><span class="n">message</span><span class="p">.</span><span class="n">text</span><span class="p">)</span>
        <span class="c1">#text_message_reply = TextSendMessage(text=
</span>        <span class="c1">#                                     f"私「{event.message.text}」\nあなた「{reply}」")
</span>        <span class="n">text_message_reply</span> <span class="o">=</span> <span class="n">TextSendMessage</span><span class="p">(</span><span class="n">text</span><span class="o">=</span><span class="n">reply</span><span class="p">)</span>

        <span class="n">messages</span> <span class="o">=</span> <span class="p">[</span><span class="n">text_message</span><span class="p">,</span> <span class="n">text_message_reply</span><span class="p">]</span>
        <span class="n">line_bot_api</span><span class="p">.</span><span class="n">reply_message</span><span class="p">(</span>
            <span class="n">event</span><span class="p">.</span><span class="n">reply_token</span><span class="p">,</span>
            <span class="n">messages</span><span class="p">)</span>

<span class="o">@</span><span class="n">handler</span><span class="p">.</span><span class="n">add</span><span class="p">(</span><span class="n">MessageEvent</span><span class="p">,</span> <span class="n">message</span><span class="o">=</span><span class="n">AudioMessage</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">handle__audio_message</span><span class="p">(</span><span class="n">event</span><span class="p">):</span>
    <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">event</span><span class="p">.</span><span class="n">message</span><span class="p">,</span> <span class="n">AudioMessage</span><span class="p">):</span>
        <span class="n">text_message</span>  <span class="o">=</span> <span class="n">TextSendMessage</span><span class="p">(</span><span class="n">text</span><span class="o">=</span><span class="s">"🤖考えています..."</span><span class="p">)</span>

        <span class="n">audio_message</span> <span class="o">=</span> <span class="n">event</span><span class="p">.</span><span class="n">message</span>
        <span class="n">audio_content</span> <span class="o">=</span> <span class="n">convert_m4a_to_wave</span><span class="p">(</span><span class="n">line_bot_api</span><span class="p">.</span><span class="n">get_message_content</span><span class="p">(</span><span class="n">audio_message</span><span class="p">.</span><span class="nb">id</span><span class="p">).</span><span class="n">content</span><span class="p">)</span>
        <span class="c1">#print("Audio content: " + str(audio_content))
</span>        <span class="c1">#print("Audio duration: " + str(audio_message.duration))
</span>        <span class="c1">#upload_wav_to_bucket(audio_content, "private")
</span>        <span class="n">transcript</span> <span class="o">=</span> <span class="n">transcribe_bytes</span><span class="p">(</span><span class="n">audio_content</span><span class="p">)</span>

        <span class="n">url</span> <span class="o">=</span> <span class="n">synthetize_text_and_upload_public</span><span class="p">(</span><span class="n">answer_filter</span><span class="p">(</span><span class="n">get_answer_from_openai_test</span><span class="p">(</span><span class="n">transcript</span><span class="p">)))</span>
        <span class="n">duration</span> <span class="o">=</span> <span class="nb">str</span><span class="p">(</span><span class="nb">int</span><span class="p">(</span><span class="nb">float</span><span class="p">(</span><span class="n">url</span><span class="p">.</span><span class="n">split</span><span class="p">(</span><span class="s">"_"</span><span class="p">)[</span><span class="mi">2</span><span class="p">])</span><span class="o">*</span><span class="mi">1000</span><span class="p">))</span> 
        <span class="n">audio_message</span> <span class="o">=</span> <span class="n">AudioSendMessage</span><span class="p">(</span><span class="n">original_content_url</span><span class="o">=</span><span class="n">url</span><span class="p">,</span> <span class="n">duration</span><span class="o">=</span><span class="n">duration</span><span class="p">)</span>

        <span class="n">messages</span> <span class="o">=</span> <span class="p">[</span><span class="n">text_message</span><span class="p">,</span> <span class="n">audio_message</span><span class="p">]</span>
        <span class="n">line_bot_api</span><span class="p">.</span><span class="n">reply_message</span><span class="p">(</span>
            <span class="n">event</span><span class="p">.</span><span class="n">reply_token</span><span class="p">,</span>
            <span class="n">messages</span><span class="p">)</span>

<span class="k">def</span> <span class="nf">sanitize</span><span class="p">(</span><span class="n">string</span><span class="p">):</span>
    <span class="c1"># Remove HTML tags
</span>    <span class="n">string</span> <span class="o">=</span> <span class="n">html</span><span class="p">.</span><span class="n">escape</span><span class="p">(</span><span class="n">string</span><span class="p">)</span>
    <span class="c1"># Remove potentially harmful characters
</span>    <span class="n">string</span> <span class="o">=</span> <span class="n">re</span><span class="p">.</span><span class="n">sub</span><span class="p">(</span><span class="sa">r</span><span class="s">'[^\w\s]+'</span><span class="p">,</span> <span class="s">''</span><span class="p">,</span> <span class="n">string</span><span class="p">)</span>
    <span class="c1"># Remove JavaScript code
</span>    <span class="n">string</span> <span class="o">=</span> <span class="n">re</span><span class="p">.</span><span class="n">sub</span><span class="p">(</span><span class="sa">r</span><span class="s">'&lt;script.*?&gt;.*?&lt;/script&gt;'</span><span class="p">,</span> <span class="s">''</span><span class="p">,</span> <span class="n">string</span><span class="p">,</span> <span class="n">flags</span><span class="o">=</span><span class="n">re</span><span class="p">.</span><span class="n">DOTALL</span> <span class="o">|</span> <span class="n">re</span><span class="p">.</span><span class="n">IGNORECASE</span><span class="p">)</span>
    <span class="c1"># Remove SQL commands
</span>    <span class="n">string</span> <span class="o">=</span> <span class="n">re</span><span class="p">.</span><span class="n">sub</span><span class="p">(</span><span class="sa">r</span><span class="s">'(\b(ALTER|CREATE|DELETE|DROP|EXEC(UTE){0,1}|INSERT( +INTO){0,1}|MERGE|SELECT|UPDATE|UNION( +ALL){0,1})\b)'</span><span class="p">,</span> <span class="s">''</span><span class="p">,</span> <span class="n">string</span><span class="p">,</span> <span class="n">flags</span><span class="o">=</span><span class="n">re</span><span class="p">.</span><span class="n">IGNORECASE</span><span class="p">)</span>
    <span class="c1"># Remove other possible harmful characters
</span>    <span class="k">return</span> <span class="n">string</span><span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"@"</span><span class="p">,</span> <span class="s">"at"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"."</span><span class="p">,</span> <span class="s">"dot"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"/"</span><span class="p">,</span> <span class="s">"slash"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"?"</span><span class="p">,</span> <span class="s">"question"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"="</span><span class="p">,</span> <span class="s">"equals"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"&amp;"</span><span class="p">,</span> <span class="s">"and"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"%"</span><span class="p">,</span> <span class="s">"percent"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"$"</span><span class="p">,</span> <span class="s">"dollar"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"#"</span><span class="p">,</span> <span class="s">"hash"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"!"</span><span class="p">,</span> <span class="s">"exclamation"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"&gt;"</span><span class="p">,</span> <span class="s">"greater"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"&lt;"</span><span class="p">,</span> <span class="s">"lesser"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">":"</span><span class="p">,</span> <span class="s">"colon"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">";"</span><span class="p">,</span> <span class="s">"semicolon"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"{"</span><span class="p">,</span> <span class="s">"braceleft"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"}"</span><span class="p">,</span> <span class="s">"braceright"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"["</span><span class="p">,</span> <span class="s">"bracketleft"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"]"</span><span class="p">,</span> <span class="s">"bracketright"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"("</span><span class="p">,</span> <span class="s">"parenleft"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">")"</span><span class="p">,</span> <span class="s">"parenright"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"'"</span><span class="p">,</span> <span class="s">"apostrophe"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">'"'</span><span class="p">,</span> <span class="s">"quotemark"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">","</span><span class="p">,</span> <span class="s">"comma"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"~"</span><span class="p">,</span> <span class="s">"tilde"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"`"</span><span class="p">,</span> <span class="s">"backtick"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"^"</span><span class="p">,</span> <span class="s">"caret"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"|"</span><span class="p">,</span> <span class="s">"verticalbar"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"</span><span class="se">\\</span><span class="s">"</span><span class="p">,</span> <span class="s">"backslash"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">" "</span><span class="p">,</span> <span class="s">"-"</span><span class="p">)</span>\
                 <span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"*"</span><span class="p">,</span> <span class="s">"asterisk"</span><span class="p">)</span>


<span class="k">def</span> <span class="nf">answer_filter</span><span class="p">(</span><span class="n">answer</span><span class="p">):</span>
    <span class="c1">#replace \n and ? with empty string
</span>    <span class="n">answer</span> <span class="o">=</span> <span class="n">answer</span><span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">"　"</span><span class="p">,</span> <span class="s">""</span><span class="p">).</span><span class="n">replace</span><span class="p">(</span><span class="s">"</span><span class="se">\n</span><span class="s">"</span><span class="p">,</span> <span class="s">""</span><span class="p">).</span><span class="n">replace</span><span class="p">(</span><span class="s">" "</span><span class="p">,</span> <span class="s">""</span><span class="p">)</span>
    <span class="c1">#split answer by "。" and return the first 2 sentences
</span>    <span class="c1">#but if there is only one sentence, return the whole answer
</span>    <span class="c1">#fragments = answer.split("。")
</span>    <span class="c1">#if len(fragments) == 1:
</span>    <span class="c1">#    return answer
</span>    <span class="c1">#elif len(fragments) &gt;= 2:
</span>    <span class="c1">#    return "。".join(fragments[:2])
</span>    <span class="c1">#elif len(fragments) &gt;= 3:
</span>    <span class="c1">#    return "。".join(fragments[:3])
</span>    <span class="k">return</span> <span class="n">answer</span>

<span class="k">def</span> <span class="nf">get_answer_from_openai_test</span><span class="p">(</span><span class="n">question</span><span class="p">):</span>
    <span class="n">prompt</span> <span class="o">=</span> <span class="n">question</span>
    
    <span class="n">response</span> <span class="o">=</span> <span class="n">openai</span><span class="p">.</span><span class="n">Completion</span><span class="p">.</span><span class="n">create</span><span class="p">(</span>
        <span class="n">engine</span><span class="o">=</span><span class="s">"text-davinci-003"</span><span class="p">,</span> 
        <span class="n">prompt</span><span class="o">=</span><span class="n">prompt</span><span class="p">,</span> 
        <span class="n">temperature</span><span class="o">=</span><span class="mf">0.5</span><span class="p">,</span>
        <span class="n">max_tokens</span><span class="o">=</span><span class="mi">250</span><span class="p">,</span> 
        <span class="n">top_p</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> 
        <span class="n">n</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span>
        <span class="n">frequency_penalty</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span> 
        <span class="n">presence_penalty</span><span class="o">=</span><span class="mi">0</span>
    <span class="p">)</span>
    <span class="k">if</span> <span class="n">response</span><span class="p">.</span><span class="n">choices</span><span class="p">:</span>
        <span class="n">answer</span> <span class="o">=</span> <span class="n">response</span><span class="p">.</span><span class="n">choices</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="n">text</span><span class="p">.</span><span class="n">strip</span><span class="p">()</span>
    <span class="k">else</span><span class="p">:</span>
        <span class="n">answer</span> <span class="o">=</span> <span class="s">"答えが見つかりませんでした"</span>
    <span class="k">return</span> <span class="n">answer</span>

<span class="k">def</span> <span class="nf">convert_m4a_to_wave</span><span class="p">(</span><span class="n">m4a_data</span><span class="p">):</span>
    <span class="n">audio</span> <span class="o">=</span> <span class="n">AudioSegment</span><span class="p">.</span><span class="n">from_file</span><span class="p">(</span><span class="n">io</span><span class="p">.</span><span class="n">BytesIO</span><span class="p">(</span><span class="n">m4a_data</span><span class="p">),</span> <span class="nb">format</span><span class="o">=</span><span class="s">'m4a'</span><span class="p">)</span>
    <span class="n">wave_data</span> <span class="o">=</span> <span class="n">audio</span><span class="p">.</span><span class="n">export</span><span class="p">(</span><span class="nb">format</span><span class="o">=</span><span class="s">'wav'</span><span class="p">).</span><span class="n">read</span><span class="p">()</span>
    <span class="k">return</span> <span class="n">wave_data</span>

<span class="k">def</span> <span class="nf">get_duration_m4a</span><span class="p">(</span><span class="n">audio_content</span><span class="p">):</span>
    <span class="n">audio_stream</span> <span class="o">=</span> <span class="n">io</span><span class="p">.</span><span class="n">BytesIO</span><span class="p">(</span><span class="n">audio_content</span><span class="p">)</span>
    <span class="n">audio</span> <span class="o">=</span> <span class="n">AudioSegment</span><span class="p">.</span><span class="n">from_file</span><span class="p">(</span><span class="n">audio_stream</span><span class="p">,</span> <span class="nb">format</span><span class="o">=</span><span class="s">'m4a'</span><span class="p">)</span>
    <span class="n">duration_ms</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="n">audio</span><span class="p">)</span>
    <span class="n">duration</span> <span class="o">=</span> <span class="n">duration_ms</span> <span class="o">/</span> <span class="mi">1000</span>
    <span class="k">return</span> <span class="n">duration</span>

<span class="k">def</span> <span class="nf">get_duration</span><span class="p">(</span><span class="n">audio_content</span><span class="p">):</span>
    <span class="k">with</span> <span class="n">io</span><span class="p">.</span><span class="n">BytesIO</span><span class="p">(</span><span class="n">audio_content</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
        <span class="k">with</span> <span class="n">wave</span><span class="p">.</span><span class="nb">open</span><span class="p">(</span><span class="n">f</span><span class="p">,</span> <span class="s">"rb"</span><span class="p">)</span> <span class="k">as</span> <span class="n">wave_file</span><span class="p">:</span>
            <span class="n">frames</span> <span class="o">=</span> <span class="n">wave_file</span><span class="p">.</span><span class="n">getnframes</span><span class="p">()</span>
            <span class="n">rate</span> <span class="o">=</span> <span class="n">wave_file</span><span class="p">.</span><span class="n">getframerate</span><span class="p">()</span>
            <span class="n">duration</span> <span class="o">=</span> <span class="n">frames</span> <span class="o">/</span> <span class="nb">float</span><span class="p">(</span><span class="n">rate</span><span class="p">)</span>
    <span class="k">return</span> <span class="n">duration</span>

<span class="k">def</span> <span class="nf">transcribe_bytes</span><span class="p">(</span><span class="n">audio_content</span><span class="p">):</span>
    <span class="n">audio_stream</span> <span class="o">=</span> <span class="n">io</span><span class="p">.</span><span class="n">BytesIO</span><span class="p">(</span><span class="n">audio_content</span><span class="p">)</span>

    <span class="c1"># Initialize the Google Cloud Speech-to-Text client
</span>    <span class="n">client</span> <span class="o">=</span> <span class="n">speech</span><span class="p">.</span><span class="n">SpeechClient</span><span class="p">()</span>

    <span class="c1"># Set the audio configuration
</span>    <span class="n">config</span> <span class="o">=</span> <span class="n">speech</span><span class="p">.</span><span class="n">RecognitionConfig</span><span class="p">(</span>
        <span class="n">encoding</span><span class="o">=</span><span class="n">speech</span><span class="p">.</span><span class="n">RecognitionConfig</span><span class="p">.</span><span class="n">AudioEncoding</span><span class="p">.</span><span class="n">LINEAR16</span><span class="p">,</span>
        <span class="n">language_code</span><span class="o">=</span><span class="s">"ja-JP"</span>
    <span class="p">)</span>

    <span class="c1"># Transcribe the audio
</span>    <span class="n">audio</span>    <span class="o">=</span> <span class="n">speech</span><span class="p">.</span><span class="n">RecognitionAudio</span><span class="p">(</span><span class="n">content</span><span class="o">=</span><span class="n">audio_stream</span><span class="p">.</span><span class="n">getvalue</span><span class="p">())</span>
    <span class="n">response</span> <span class="o">=</span> <span class="n">client</span><span class="p">.</span><span class="n">recognize</span><span class="p">(</span><span class="n">config</span><span class="o">=</span><span class="n">config</span><span class="p">,</span> <span class="n">audio</span><span class="o">=</span><span class="n">audio</span><span class="p">)</span>

    <span class="k">return</span> <span class="n">response</span><span class="p">.</span><span class="n">results</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="n">alternatives</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="n">transcript</span>

<span class="k">def</span> <span class="nf">synthetize_text</span><span class="p">(</span><span class="n">t</span><span class="p">):</span>
    <span class="n">client</span> <span class="o">=</span> <span class="n">texttospeech</span><span class="p">.</span><span class="n">TextToSpeechClient</span><span class="p">()</span>

    <span class="c1"># Set the text input to be synthesized
</span>    <span class="n">synthesis_input</span> <span class="o">=</span> <span class="n">texttospeech</span><span class="p">.</span><span class="n">SynthesisInput</span><span class="p">(</span><span class="n">text</span><span class="o">=</span><span class="n">t</span><span class="p">)</span>

    <span class="c1"># Build the voice request, select the language code ("en-US") and the ssml
</span>    <span class="c1"># voice gender ("neutral")
</span>    <span class="n">voice</span> <span class="o">=</span> <span class="n">texttospeech</span><span class="p">.</span><span class="n">VoiceSelectionParams</span><span class="p">(</span>
        <span class="n">language_code</span><span class="o">=</span><span class="s">"ja-JP"</span><span class="p">,</span> <span class="n">ssml_gender</span><span class="o">=</span><span class="n">texttospeech</span><span class="p">.</span><span class="n">SsmlVoiceGender</span><span class="p">.</span><span class="n">NEUTRAL</span>
    <span class="p">)</span>

    <span class="c1"># Select the type of audio file you want returned
</span>    <span class="n">audio_config</span> <span class="o">=</span> <span class="n">texttospeech</span><span class="p">.</span><span class="n">AudioConfig</span><span class="p">(</span>
        <span class="n">audio_encoding</span><span class="o">=</span><span class="n">texttospeech</span><span class="p">.</span><span class="n">AudioEncoding</span><span class="p">.</span><span class="n">LINEAR16</span>
    <span class="p">)</span>

    <span class="c1"># Perform the text-to-speech request on the text input with the selected
</span>    <span class="c1"># voice parameters and audio file type
</span>    <span class="n">response</span> <span class="o">=</span> <span class="n">client</span><span class="p">.</span><span class="n">synthesize_speech</span><span class="p">(</span>
        <span class="nb">input</span><span class="o">=</span><span class="n">synthesis_input</span><span class="p">,</span> <span class="n">voice</span><span class="o">=</span><span class="n">voice</span><span class="p">,</span> <span class="n">audio_config</span><span class="o">=</span><span class="n">audio_config</span>
    <span class="p">)</span>

    <span class="c1"># Return the audio content
</span>    <span class="k">return</span> <span class="n">response</span><span class="p">.</span><span class="n">audio_content</span>

<span class="k">def</span> <span class="nf">upload_wav_to_bucket</span><span class="p">(</span><span class="n">audio_content</span><span class="p">,</span> <span class="n">sharedType</span><span class="p">):</span>
    <span class="c1"># Calculate the duration of the audio
</span>    <span class="n">duration</span> <span class="o">=</span> <span class="n">get_duration</span><span class="p">(</span><span class="n">audio_content</span><span class="p">)</span>

    <span class="c1"># Construct the file name
</span>    <span class="n">uuidstr</span> <span class="o">=</span> <span class="nb">str</span><span class="p">(</span><span class="n">uuid</span><span class="p">.</span><span class="n">uuid4</span><span class="p">())</span>
    <span class="k">if</span> <span class="n">sharedType</span> <span class="o">==</span> <span class="s">"public"</span><span class="p">:</span>
        <span class="n">file_name</span> <span class="o">=</span> <span class="sa">f</span><span class="s">"public_</span><span class="si">{</span><span class="n">uuidstr</span><span class="si">}</span><span class="s">_</span><span class="si">{</span><span class="n">duration</span><span class="si">}</span><span class="s">_.wav"</span>
    <span class="k">elif</span> <span class="n">sharedType</span> <span class="o">==</span> <span class="s">"temporary"</span><span class="p">:</span>
        <span class="n">file_name</span> <span class="o">=</span> <span class="sa">f</span><span class="s">"tempor_</span><span class="si">{</span><span class="n">uuidstr</span><span class="si">}</span><span class="s">_</span><span class="si">{</span><span class="n">duration</span><span class="si">}</span><span class="s">_.wav"</span>
    <span class="k">else</span><span class="p">:</span>
        <span class="n">file_name</span> <span class="o">=</span> <span class="sa">f</span><span class="s">"privat_</span><span class="si">{</span><span class="n">uuidstr</span><span class="si">}</span><span class="s">_</span><span class="si">{</span><span class="n">duration</span><span class="si">}</span><span class="s">_.wav"</span>

    <span class="c1"># Upload the audio content to the Google Cloud Storage bucket
</span>    <span class="n">storage_client</span> <span class="o">=</span> <span class="n">storage</span><span class="p">.</span><span class="n">Client</span><span class="p">()</span>
    <span class="n">bucket</span> <span class="o">=</span> <span class="n">storage_client</span><span class="p">.</span><span class="n">bucket</span><span class="p">(</span><span class="s">"{}"</span><span class="p">.</span><span class="nb">format</span><span class="p">(</span><span class="n">BUCKET_NAME</span><span class="p">))</span>
    <span class="n">blob</span> <span class="o">=</span> <span class="n">bucket</span><span class="p">.</span><span class="n">blob</span><span class="p">(</span><span class="n">file_name</span><span class="p">)</span>
    <span class="n">blob</span><span class="p">.</span><span class="n">upload_from_string</span><span class="p">(</span><span class="n">audio_content</span><span class="p">,</span> <span class="n">content_type</span><span class="o">=</span><span class="s">"audio/wav"</span><span class="p">)</span>

    <span class="k">if</span> <span class="n">sharedType</span> <span class="o">==</span> <span class="s">"public"</span><span class="p">:</span>
        <span class="c1"># Make the uploaded file public
</span>        <span class="n">blob</span><span class="p">.</span><span class="n">make_public</span><span class="p">()</span>
        <span class="k">return</span> <span class="n">blob</span><span class="p">.</span><span class="n">public_url</span>
    <span class="k">elif</span> <span class="n">sharedType</span> <span class="o">==</span> <span class="s">"temporary"</span><span class="p">:</span>
        <span class="k">return</span>  <span class="n">blob</span><span class="p">.</span><span class="n">generate_signed_url</span><span class="p">(</span><span class="n">datetime</span><span class="p">.</span><span class="n">timedelta</span><span class="p">(</span><span class="n">minutes</span><span class="o">=</span><span class="mi">30</span><span class="p">))</span>
    <span class="k">else</span><span class="p">:</span>
        <span class="k">return</span> <span class="sa">f</span><span class="s">"https://storage.googleapis.com/</span><span class="si">{</span><span class="n">BUCKET_NAME</span><span class="si">}</span><span class="s">/</span><span class="si">{</span><span class="n">file_name</span><span class="si">}</span><span class="s">"</span>

<span class="k">def</span> <span class="nf">synthetize_text_and_upload_public</span><span class="p">(</span><span class="n">t</span><span class="p">):</span>
    <span class="c1"># Synthetize the text
</span>    <span class="n">t</span> <span class="o">=</span> <span class="n">sanitize</span><span class="p">(</span><span class="n">t</span><span class="p">)</span>
    <span class="n">audio_content</span> <span class="o">=</span> <span class="n">synthetize_text</span><span class="p">(</span><span class="n">t</span><span class="p">)</span>
    <span class="k">return</span> <span class="n">upload_wav_to_bucket</span><span class="p">(</span><span class="n">audio_content</span><span class="p">,</span> <span class="s">"public"</span><span class="p">)</span>

<span class="k">def</span> <span class="nf">synthetize_text_and_upload_temp</span><span class="p">(</span><span class="n">t</span><span class="p">):</span>
    <span class="c1"># Synthetize the text
</span>    <span class="n">t</span> <span class="o">=</span> <span class="n">sanitize</span><span class="p">(</span><span class="n">t</span><span class="p">)</span>
    <span class="n">audio_content</span> <span class="o">=</span> <span class="n">synthetize_text</span><span class="p">(</span><span class="n">t</span><span class="p">)</span>
    <span class="k">return</span> <span class="n">upload_wav_to_bucket</span><span class="p">(</span><span class="n">audio_content</span><span class="p">,</span> <span class="s">"temporary"</span><span class="p">)</span>

<span class="k">def</span> <span class="nf">synthetize_text_and_upload</span><span class="p">(</span><span class="n">t</span><span class="p">):</span>
    <span class="c1"># Synthetize the text
</span>    <span class="n">t</span> <span class="o">=</span> <span class="n">sanitize</span><span class="p">(</span><span class="n">t</span><span class="p">)</span>
    <span class="n">audio_content</span> <span class="o">=</span> <span class="n">synthetize_text</span><span class="p">(</span><span class="n">t</span><span class="p">)</span>
    <span class="k">return</span> <span class="n">upload_wav_to_bucket</span><span class="p">(</span><span class="n">audio_content</span><span class="p">,</span> <span class="s">"private"</span><span class="p">)</span>
</code></pre></div></div>

<h4 id="requirementstxt-1">requirements.txt</h4>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">aiohttp</span><span class="o">==</span><span class="mf">3.8</span><span class="p">.</span><span class="mi">3</span>
<span class="n">aiosignal</span><span class="o">==</span><span class="mf">1.3</span><span class="p">.</span><span class="mi">1</span>
<span class="k">async</span><span class="o">-</span><span class="n">timeout</span><span class="o">==</span><span class="mf">4.0</span><span class="p">.</span><span class="mi">2</span>
<span class="n">attrs</span><span class="o">==</span><span class="mf">22.2</span><span class="p">.</span><span class="mi">0</span>
<span class="n">beautifulsoup4</span><span class="o">==</span><span class="mf">4.11</span><span class="p">.</span><span class="mi">2</span>
<span class="n">cachetools</span><span class="o">==</span><span class="mf">5.3</span><span class="p">.</span><span class="mi">0</span>
<span class="n">certifi</span><span class="o">==</span><span class="mf">2022.12</span><span class="p">.</span><span class="mi">7</span>
<span class="n">cffi</span><span class="o">==</span><span class="mf">1.15</span><span class="p">.</span><span class="mi">1</span>
<span class="n">charset</span><span class="o">-</span><span class="n">normalizer</span><span class="o">==</span><span class="mf">2.1</span><span class="p">.</span><span class="mi">1</span>
<span class="n">click</span><span class="o">==</span><span class="mf">8.1</span><span class="p">.</span><span class="mi">3</span>
<span class="n">cloudevents</span><span class="o">==</span><span class="mf">1.9</span><span class="p">.</span><span class="mi">0</span>
<span class="n">deprecation</span><span class="o">==</span><span class="mf">2.1</span><span class="p">.</span><span class="mi">0</span>
<span class="n">Flask</span><span class="o">==</span><span class="mf">2.2</span><span class="p">.</span><span class="mi">2</span>
<span class="n">frozenlist</span><span class="o">==</span><span class="mf">1.3</span><span class="p">.</span><span class="mi">3</span>
<span class="n">functions</span><span class="o">-</span><span class="n">framework</span><span class="o">==</span><span class="mf">3.3</span><span class="p">.</span><span class="mi">0</span>
<span class="n">future</span><span class="o">==</span><span class="mf">0.18</span><span class="p">.</span><span class="mi">3</span>
<span class="n">google</span><span class="o">==</span><span class="mf">3.0</span><span class="p">.</span><span class="mi">0</span>
<span class="n">google</span><span class="o">-</span><span class="n">api</span><span class="o">-</span><span class="n">core</span><span class="o">==</span><span class="mf">2.11</span><span class="p">.</span><span class="mi">0</span>
<span class="n">google</span><span class="o">-</span><span class="n">api</span><span class="o">-</span><span class="n">python</span><span class="o">-</span><span class="n">client</span><span class="o">==</span><span class="mf">2.78</span><span class="p">.</span><span class="mi">0</span>
<span class="n">google</span><span class="o">-</span><span class="n">auth</span><span class="o">==</span><span class="mf">2.16</span><span class="p">.</span><span class="mi">0</span>
<span class="n">google</span><span class="o">-</span><span class="n">auth</span><span class="o">-</span><span class="n">httplib2</span><span class="o">==</span><span class="mf">0.1</span><span class="p">.</span><span class="mi">0</span>
<span class="n">google</span><span class="o">-</span><span class="n">cloud</span><span class="o">==</span><span class="mf">0.34</span><span class="p">.</span><span class="mi">0</span>
<span class="n">google</span><span class="o">-</span><span class="n">cloud</span><span class="o">-</span><span class="n">core</span><span class="o">==</span><span class="mf">2.3</span><span class="p">.</span><span class="mi">2</span>
<span class="n">google</span><span class="o">-</span><span class="n">cloud</span><span class="o">-</span><span class="n">speech</span><span class="o">==</span><span class="mf">2.17</span><span class="p">.</span><span class="mi">3</span>
<span class="n">google</span><span class="o">-</span><span class="n">cloud</span><span class="o">-</span><span class="n">storage</span><span class="o">==</span><span class="mf">2.7</span><span class="p">.</span><span class="mi">0</span>
<span class="n">google</span><span class="o">-</span><span class="n">cloud</span><span class="o">-</span><span class="n">texttospeech</span><span class="o">==</span><span class="mf">2.14</span><span class="p">.</span><span class="mi">1</span>
<span class="n">google</span><span class="o">-</span><span class="n">crc32c</span><span class="o">==</span><span class="mf">1.5</span><span class="p">.</span><span class="mi">0</span>
<span class="n">google</span><span class="o">-</span><span class="n">resumable</span><span class="o">-</span><span class="n">media</span><span class="o">==</span><span class="mf">2.4</span><span class="p">.</span><span class="mi">1</span>
<span class="n">googleapis</span><span class="o">-</span><span class="n">common</span><span class="o">-</span><span class="n">protos</span><span class="o">==</span><span class="mf">1.58</span><span class="p">.</span><span class="mi">0</span>
<span class="n">grpcio</span><span class="o">==</span><span class="mf">1.51</span><span class="p">.</span><span class="mi">1</span>
<span class="n">grpcio</span><span class="o">-</span><span class="n">status</span><span class="o">==</span><span class="mf">1.51</span><span class="p">.</span><span class="mi">1</span>
<span class="n">gunicorn</span><span class="o">==</span><span class="mf">20.1</span><span class="p">.</span><span class="mi">0</span>
<span class="n">httplib2</span><span class="o">==</span><span class="mf">0.21</span><span class="p">.</span><span class="mi">0</span>
<span class="n">idna</span><span class="o">==</span><span class="mf">3.4</span>
<span class="n">itsdangerous</span><span class="o">==</span><span class="mf">2.1</span><span class="p">.</span><span class="mi">2</span>
<span class="n">Jinja2</span><span class="o">==</span><span class="mf">3.1</span><span class="p">.</span><span class="mi">2</span>
<span class="n">line</span><span class="o">-</span><span class="n">bot</span><span class="o">-</span><span class="n">sdk</span><span class="o">==</span><span class="mf">2.4</span><span class="p">.</span><span class="mi">1</span>
<span class="n">MarkupSafe</span><span class="o">==</span><span class="mf">2.1</span><span class="p">.</span><span class="mi">2</span>
<span class="n">multidict</span><span class="o">==</span><span class="mf">6.0</span><span class="p">.</span><span class="mi">4</span>
<span class="n">openai</span><span class="o">==</span><span class="mf">0.26</span><span class="p">.</span><span class="mi">5</span>
<span class="n">packaging</span><span class="o">==</span><span class="mf">23.0</span>
<span class="n">proto</span><span class="o">-</span><span class="n">plus</span><span class="o">==</span><span class="mf">1.22</span><span class="p">.</span><span class="mi">2</span>
<span class="n">protobuf</span><span class="o">==</span><span class="mf">4.21</span><span class="p">.</span><span class="mi">12</span>
<span class="n">pyasn1</span><span class="o">==</span><span class="mf">0.4</span><span class="p">.</span><span class="mi">8</span>
<span class="n">pyasn1</span><span class="o">-</span><span class="n">modules</span><span class="o">==</span><span class="mf">0.2</span><span class="p">.</span><span class="mi">8</span>
<span class="n">pycparser</span><span class="o">==</span><span class="mf">2.21</span>
<span class="n">pydub</span><span class="o">==</span><span class="mf">0.25</span><span class="p">.</span><span class="mi">1</span>
<span class="n">pyparsing</span><span class="o">==</span><span class="mf">3.0</span><span class="p">.</span><span class="mi">9</span>
<span class="n">python</span><span class="o">-</span><span class="n">dotenv</span><span class="o">==</span><span class="mf">0.21</span><span class="p">.</span><span class="mi">1</span>
<span class="n">requests</span><span class="o">==</span><span class="mf">2.28</span><span class="p">.</span><span class="mi">1</span>
<span class="n">rsa</span><span class="o">==</span><span class="mf">4.9</span>
<span class="n">six</span><span class="o">==</span><span class="mf">1.16</span><span class="p">.</span><span class="mi">0</span>
<span class="n">soundfile</span><span class="o">==</span><span class="mf">0.12</span><span class="p">.</span><span class="mi">1</span>
<span class="n">soupsieve</span><span class="o">==</span><span class="mf">2.4</span>
<span class="n">tqdm</span><span class="o">==</span><span class="mf">4.64</span><span class="p">.</span><span class="mi">1</span>
<span class="n">uritemplate</span><span class="o">==</span><span class="mf">4.1</span><span class="p">.</span><span class="mi">1</span>
<span class="n">urllib3</span><span class="o">==</span><span class="mf">1.26</span><span class="p">.</span><span class="mi">14</span>
<span class="n">watchdog</span><span class="o">==</span><span class="mf">2.2</span><span class="p">.</span><span class="mi">1</span>
<span class="n">Werkzeug</span><span class="o">==</span><span class="mf">2.2</span><span class="p">.</span><span class="mi">3</span>
<span class="n">yarl</span><span class="o">==</span><span class="mf">1.8</span><span class="p">.</span><span class="mi">2</span>
</code></pre></div></div>]]></content><author><name>Tomohiko Araki</name></author><summary type="html"><![CDATA[A LINE bot for generating pixel art, coded by python and running in CloudFunctions.]]></summary></entry><entry><title type="html">Convert your voice to Zundamon</title><link href="https://metaproxy.co/Zundamon/" rel="alternate" type="text/html" title="Convert your voice to Zundamon" /><published>2022-12-18T00:00:00+00:00</published><updated>2022-12-18T00:00:00+00:00</updated><id>https://metaproxy.co/Zundamon</id><content type="html" xml:base="https://metaproxy.co/Zundamon/"><![CDATA[<p>VOICEVOX make to feel better, as your voice is converted to cute Zundamon. In this article I am going to show to how to do it; the result is clutterted and crappy still due to my lack of audio processing skill, but this just works.</p>

<h2 id="how-it-works">How it works</h2>

<p>Thanks for built-in functionality of REST API of VOICEBOX, you can easily convert your voice to Zundamon.</p>

<ul>
  <li>This method is to convert your voice as transcript using Google Speech API and then convert the transcript to Zundamon using VOICEVOX</li>
  <li>Huge latency and no proper datastream is implemented, but it just works.</li>
  <li>Used named pipe for internal communication.</li>
</ul>

<pre><code class="language-mermaid">graph LR;
  A[Mic];
  B[Voice to text converter];
  G[Google speech]
  C[Text to Voice converter];
  D[VOICEBOX];
  E[Audio file];
  F[Speaker]

  A -- your voice --&gt; B
  B -- sound --&gt; G
  G -- text  --&gt; B
  B -- transcript --&gt; C
  C -- REST API   --&gt; D
  D -- wav file   --&gt; C
  C --&gt; F

  subgraph Generate transcript;
    B
    G
  end

  subgraph Generate Zundamon voice;
    C
    D
  end

</code></pre>

<h2 id="updated-on-2023-02-13">(Updated on 2023-02-13)</h2>

<p>I have found good example for retrieving input from microphone as stream, so rewrite codes.
I think performance would be more efficient if I could take following:</p>
<ul>
  <li>Decom play.sh, write synthetized wav into sound device directly and concurrently.</li>
  <li>Prepare good performance machine for VOICEVOX on cloud like cloudRun or on-prem.</li>
  <li>Make connection to VOICEVOX realtime-mannered.</li>
</ul>

<h2 id="implementation">Implementation</h2>

<h3 id="voicevox-installation">VOICEVOX installation</h3>

<p>Just download and install it.
<a href="https://voicevox.hiroshiba.jp/">https://voicevox.hiroshiba.jp/</a></p>

<p>Thanks to Hiho-san that VOICEBOX has built-in REST API so just executing binary provides the entry point of API in localhost.</p>

<h3 id="prepare-playsh-for-playing-synthetized-voice-file">Prepare play.sh for playing synthetized voice file.</h3>

<p>This is the messy part of this trial. Reliant to file system to signaling of synthetize has been completed, and trying to play wavs as fast as possible. Yes I should have direct and concurrent channel to audio devices.</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">while </span><span class="nb">true</span><span class="p">;</span> <span class="k">do</span>
 <span class="c"># Play oldest first, and rename to .played</span>
 <span class="nb">ls</span> <span class="nt">-tr</span> <span class="k">*</span>.wav 2&gt;/dev/null | xargs <span class="nt">-I</span> <span class="o">{}</span> <span class="nb">echo</span> <span class="nt">-e</span> <span class="s2">"</span><span class="se">\n</span><span class="s2">"</span><span class="o">{}</span> | xargs <span class="nt">-d</span> <span class="s1">$'</span><span class="se">\n</span><span class="s1">'</span> sh <span class="nt">-c</span> <span class="s1">'for arg; do if [ -n "$arg" ]; then aplay "$arg"; mv "$arg" "$arg".played; fi; done'</span>
 <span class="nb">ls</span> <span class="nt">-t</span> <span class="k">*</span>.wav.played 2&gt;/dev/null | <span class="nb">tail</span> <span class="nt">-1</span> | xargs <span class="nt">-I</span> <span class="o">{}</span> <span class="nb">rm</span> <span class="o">{}</span>
 <span class="nb">sleep </span>0.01
<span class="k">done</span>
</code></pre></div></div>

<h3 id="prepare-voice-transcript-synthesizedpy-for-forging-synthetized-cute-voice-file-from-your-real-voice">Prepare voice-transcript-synthesized.py for forging synthetized cute voice file from your real voice.</h3>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># from https://cloud.google.com/speech-to-text/docs/samples/speech-transcribe-streaming-mic
</span><span class="kn">from</span> <span class="nn">__future__</span> <span class="kn">import</span> <span class="n">division</span>

<span class="kn">import</span> <span class="nn">re</span>
<span class="kn">import</span> <span class="nn">sys</span>
<span class="kn">import</span> <span class="nn">uuid</span>
<span class="kn">import</span> <span class="nn">_thread</span>

<span class="kn">from</span> <span class="nn">google.cloud</span> <span class="kn">import</span> <span class="n">speech</span>
<span class="kn">from</span> <span class="nn">google.cloud</span> <span class="kn">import</span> <span class="n">texttospeech</span>

<span class="kn">import</span> <span class="nn">pyaudio</span>
<span class="kn">from</span> <span class="nn">six.moves</span> <span class="kn">import</span> <span class="n">queue</span>

<span class="kn">import</span> <span class="nn">requests</span>
<span class="kn">import</span> <span class="nn">argparse</span>
<span class="kn">import</span> <span class="nn">json</span>

<span class="c1"># Audio recording parameters
</span><span class="n">RATE</span> <span class="o">=</span> <span class="mi">16000</span>
<span class="n">CHUNK</span> <span class="o">=</span> <span class="nb">int</span><span class="p">(</span><span class="n">RATE</span> <span class="o">/</span> <span class="mi">10</span><span class="p">)</span>  <span class="c1"># 100ms
</span>
<span class="n">VOICEVOX_HOST</span><span class="o">=</span><span class="s">'localhost'</span>
<span class="n">SYNTHETIZE_BY</span><span class="o">=</span><span class="s">'vox'</span> <span class="c1"># google or vox
</span>
<span class="k">class</span> <span class="nc">MicrophoneStream</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
    <span class="s">"""Opens a recording stream as a generator yielding the audio chunks."""</span>

    <span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">rate</span><span class="p">,</span> <span class="n">chunk</span><span class="p">):</span>
        <span class="bp">self</span><span class="p">.</span><span class="n">_rate</span> <span class="o">=</span> <span class="n">rate</span>
        <span class="bp">self</span><span class="p">.</span><span class="n">_chunk</span> <span class="o">=</span> <span class="n">chunk</span>

        <span class="c1"># Create a thread-safe buffer of audio data
</span>        <span class="bp">self</span><span class="p">.</span><span class="n">_buff</span> <span class="o">=</span> <span class="n">queue</span><span class="p">.</span><span class="n">Queue</span><span class="p">()</span>
        <span class="bp">self</span><span class="p">.</span><span class="n">closed</span> <span class="o">=</span> <span class="bp">True</span>

    <span class="k">def</span> <span class="nf">__enter__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
        <span class="bp">self</span><span class="p">.</span><span class="n">_audio_interface</span> <span class="o">=</span> <span class="n">pyaudio</span><span class="p">.</span><span class="n">PyAudio</span><span class="p">()</span>
        <span class="bp">self</span><span class="p">.</span><span class="n">_audio_stream</span> <span class="o">=</span> <span class="bp">self</span><span class="p">.</span><span class="n">_audio_interface</span><span class="p">.</span><span class="nb">open</span><span class="p">(</span>
            <span class="nb">format</span><span class="o">=</span><span class="n">pyaudio</span><span class="p">.</span><span class="n">paInt16</span><span class="p">,</span>
            <span class="c1"># The API currently only supports 1-channel (mono) audio
</span>            <span class="c1"># https://goo.gl/z757pE
</span>            <span class="n">channels</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span>
            <span class="n">rate</span><span class="o">=</span><span class="bp">self</span><span class="p">.</span><span class="n">_rate</span><span class="p">,</span>
            <span class="nb">input</span><span class="o">=</span><span class="bp">True</span><span class="p">,</span>
            <span class="n">frames_per_buffer</span><span class="o">=</span><span class="bp">self</span><span class="p">.</span><span class="n">_chunk</span><span class="p">,</span>
            <span class="c1"># Run the audio stream asynchronously to fill the buffer object.
</span>            <span class="c1"># This is necessary so that the input device's buffer doesn't
</span>            <span class="c1"># overflow while the calling thread makes network requests, etc.
</span>            <span class="n">stream_callback</span><span class="o">=</span><span class="bp">self</span><span class="p">.</span><span class="n">_fill_buffer</span><span class="p">,</span>
        <span class="p">)</span>

        <span class="bp">self</span><span class="p">.</span><span class="n">closed</span> <span class="o">=</span> <span class="bp">False</span>

        <span class="k">return</span> <span class="bp">self</span>

    <span class="k">def</span> <span class="nf">__exit__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="nb">type</span><span class="p">,</span> <span class="n">value</span><span class="p">,</span> <span class="n">traceback</span><span class="p">):</span>
        <span class="bp">self</span><span class="p">.</span><span class="n">_audio_stream</span><span class="p">.</span><span class="n">stop_stream</span><span class="p">()</span>
        <span class="bp">self</span><span class="p">.</span><span class="n">_audio_stream</span><span class="p">.</span><span class="n">close</span><span class="p">()</span>
        <span class="bp">self</span><span class="p">.</span><span class="n">closed</span> <span class="o">=</span> <span class="bp">True</span>
        <span class="c1"># Signal the generator to terminate so that the client's
</span>        <span class="c1"># streaming_recognize method will not block the process termination.
</span>        <span class="bp">self</span><span class="p">.</span><span class="n">_buff</span><span class="p">.</span><span class="n">put</span><span class="p">(</span><span class="bp">None</span><span class="p">)</span>
        <span class="bp">self</span><span class="p">.</span><span class="n">_audio_interface</span><span class="p">.</span><span class="n">terminate</span><span class="p">()</span>

    <span class="k">def</span> <span class="nf">_fill_buffer</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">in_data</span><span class="p">,</span> <span class="n">frame_count</span><span class="p">,</span> <span class="n">time_info</span><span class="p">,</span> <span class="n">status_flags</span><span class="p">):</span>
        <span class="s">"""Continuously collect data from the audio stream, into the buffer."""</span>
        <span class="bp">self</span><span class="p">.</span><span class="n">_buff</span><span class="p">.</span><span class="n">put</span><span class="p">(</span><span class="n">in_data</span><span class="p">)</span>
        <span class="k">return</span> <span class="bp">None</span><span class="p">,</span> <span class="n">pyaudio</span><span class="p">.</span><span class="n">paContinue</span>

    <span class="k">def</span> <span class="nf">generator</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
        <span class="k">while</span> <span class="ow">not</span> <span class="bp">self</span><span class="p">.</span><span class="n">closed</span><span class="p">:</span>
            <span class="c1"># Use a blocking get() to ensure there's at least one chunk of
</span>            <span class="c1"># data, and stop iteration if the chunk is None, indicating the
</span>            <span class="c1"># end of the audio stream.
</span>            <span class="n">chunk</span> <span class="o">=</span> <span class="bp">self</span><span class="p">.</span><span class="n">_buff</span><span class="p">.</span><span class="n">get</span><span class="p">()</span>
            <span class="k">if</span> <span class="n">chunk</span> <span class="ow">is</span> <span class="bp">None</span><span class="p">:</span>
                <span class="k">return</span>
            <span class="n">data</span> <span class="o">=</span> <span class="p">[</span><span class="n">chunk</span><span class="p">]</span>

            <span class="c1"># Now consume whatever other data's still buffered.
</span>            <span class="k">while</span> <span class="bp">True</span><span class="p">:</span>
                <span class="k">try</span><span class="p">:</span>
                    <span class="n">chunk</span> <span class="o">=</span> <span class="bp">self</span><span class="p">.</span><span class="n">_buff</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="n">block</span><span class="o">=</span><span class="bp">False</span><span class="p">)</span>
                    <span class="k">if</span> <span class="n">chunk</span> <span class="ow">is</span> <span class="bp">None</span><span class="p">:</span>
                        <span class="k">return</span>
                    <span class="n">data</span><span class="p">.</span><span class="n">append</span><span class="p">(</span><span class="n">chunk</span><span class="p">)</span>
                <span class="k">except</span> <span class="n">queue</span><span class="p">.</span><span class="n">Empty</span><span class="p">:</span>
                    <span class="k">break</span>

            <span class="k">yield</span> <span class="sa">b</span><span class="s">""</span><span class="p">.</span><span class="n">join</span><span class="p">(</span><span class="n">data</span><span class="p">)</span>


<span class="k">def</span> <span class="nf">listen_print_loop</span><span class="p">(</span><span class="n">responses</span><span class="p">):</span>
    <span class="s">"""Iterates through server responses and prints them.

    The responses passed is a generator that will block until a response
    is provided by the server.

    Each response may contain multiple results, and each result may contain
    multiple alternatives; for details, see https://goo.gl/tjCPAU.  Here we
    print only the transcription for the top alternative of the top result.

    In this case, responses are provided for interim results as well. If the
    response is an interim one, print a line feed at the end of it, to allow
    the next result to overwrite it, until the response is a final one. For the
    final one, print a newline to preserve the finalized transcription.
    """</span>
    <span class="n">num_chars_printed</span> <span class="o">=</span> <span class="mi">0</span>
    <span class="k">for</span> <span class="n">response</span> <span class="ow">in</span> <span class="n">responses</span><span class="p">:</span>
        <span class="k">if</span> <span class="ow">not</span> <span class="n">response</span><span class="p">.</span><span class="n">results</span><span class="p">:</span>
            <span class="k">continue</span>

        <span class="c1"># The `results` list is consecutive. For streaming, we only care about
</span>        <span class="c1"># the first result being considered, since once it's `is_final`, it
</span>        <span class="c1"># moves on to considering the next utterance.
</span>        <span class="n">result</span> <span class="o">=</span> <span class="n">response</span><span class="p">.</span><span class="n">results</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
        <span class="k">if</span> <span class="ow">not</span> <span class="n">result</span><span class="p">.</span><span class="n">alternatives</span><span class="p">:</span>
            <span class="k">continue</span>

        <span class="c1"># Display the transcription of the top alternative.
</span>        <span class="n">transcript</span> <span class="o">=</span> <span class="n">result</span><span class="p">.</span><span class="n">alternatives</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="n">transcript</span>

        <span class="c1"># Display interim results, but with a carriage return at the end of the
</span>        <span class="c1"># line, so subsequent lines will overwrite them.
</span>        <span class="c1">#
</span>        <span class="c1"># If the previous result was longer than this one, we need to print
</span>        <span class="c1"># some extra spaces to overwrite the previous result
</span>        <span class="n">overwrite_chars</span> <span class="o">=</span> <span class="s">" "</span> <span class="o">*</span> <span class="p">(</span><span class="n">num_chars_printed</span> <span class="o">-</span> <span class="nb">len</span><span class="p">(</span><span class="n">transcript</span><span class="p">))</span>

        <span class="k">if</span> <span class="ow">not</span> <span class="n">result</span><span class="p">.</span><span class="n">is_final</span><span class="p">:</span>
            <span class="n">sys</span><span class="p">.</span><span class="n">stdout</span><span class="p">.</span><span class="n">write</span><span class="p">(</span><span class="n">transcript</span> <span class="o">+</span> <span class="n">overwrite_chars</span> <span class="o">+</span> <span class="s">"</span><span class="se">\r</span><span class="s">"</span><span class="p">)</span>
            <span class="n">sys</span><span class="p">.</span><span class="n">stdout</span><span class="p">.</span><span class="n">flush</span><span class="p">()</span>

            <span class="n">num_chars_printed</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="n">transcript</span><span class="p">)</span>

        <span class="k">else</span><span class="p">:</span>
            <span class="k">print</span><span class="p">(</span><span class="n">transcript</span> <span class="o">+</span> <span class="n">overwrite_chars</span> <span class="o">+</span> <span class="s">" [CONFIRMED]"</span><span class="p">)</span>
            <span class="k">try</span><span class="p">:</span>
              <span class="k">if</span> <span class="n">SYNTHETIZE_BY</span><span class="o">==</span><span class="s">'google'</span><span class="p">:</span>
                <span class="n">_thread</span><span class="p">.</span><span class="n">start_new_thread</span><span class="p">(</span> <span class="n">synthetize_text</span><span class="p">,</span> <span class="p">(</span><span class="n">transcript</span><span class="p">,)</span> <span class="p">)</span>
              <span class="k">elif</span> <span class="n">SYNTHETIZE_BY</span><span class="o">==</span><span class="s">'vox'</span><span class="p">:</span>
                <span class="n">_thread</span><span class="p">.</span><span class="n">start_new_thread</span><span class="p">(</span> <span class="n">synthetizeZ_text</span><span class="p">,</span> <span class="p">(</span><span class="n">transcript</span><span class="p">,)</span> <span class="p">)</span>
            <span class="k">except</span><span class="p">:</span>
              <span class="k">print</span><span class="p">(</span><span class="s">"Error: unable to start thread"</span><span class="p">)</span>

            <span class="c1"># Exit recognition if any of the transcribed phrases could be
</span>            <span class="c1"># one of our keywords.
</span>            <span class="k">if</span> <span class="n">re</span><span class="p">.</span><span class="n">search</span><span class="p">(</span><span class="sa">r</span><span class="s">"\b(exit|quit)\b"</span><span class="p">,</span> <span class="n">transcript</span><span class="p">,</span> <span class="n">re</span><span class="p">.</span><span class="n">I</span><span class="p">):</span>
                <span class="k">print</span><span class="p">(</span><span class="s">"Exiting.."</span><span class="p">)</span>
                <span class="k">break</span>

            <span class="n">num_chars_printed</span> <span class="o">=</span> <span class="mi">0</span>

<span class="k">def</span> <span class="nf">synthetize_text</span><span class="p">(</span><span class="n">t</span><span class="p">):</span>
    <span class="n">client</span> <span class="o">=</span> <span class="n">texttospeech</span><span class="p">.</span><span class="n">TextToSpeechClient</span><span class="p">()</span>

    <span class="c1"># Set the text input to be synthesized
</span>    <span class="n">synthesis_input</span> <span class="o">=</span> <span class="n">texttospeech</span><span class="p">.</span><span class="n">SynthesisInput</span><span class="p">(</span><span class="n">text</span><span class="o">=</span><span class="n">t</span><span class="p">)</span>

    <span class="c1"># Build the voice request, select the language code ("en-US") and the ssml
</span>    <span class="c1"># voice gender ("neutral")
</span>    <span class="n">voice</span> <span class="o">=</span> <span class="n">texttospeech</span><span class="p">.</span><span class="n">VoiceSelectionParams</span><span class="p">(</span>
        <span class="n">language_code</span><span class="o">=</span><span class="s">"ja-JP"</span><span class="p">,</span> <span class="n">ssml_gender</span><span class="o">=</span><span class="n">texttospeech</span><span class="p">.</span><span class="n">SsmlVoiceGender</span><span class="p">.</span><span class="n">NEUTRAL</span>
    <span class="p">)</span>

    <span class="c1"># Select the type of audio file you want returned
</span>    <span class="n">audio_config</span> <span class="o">=</span> <span class="n">texttospeech</span><span class="p">.</span><span class="n">AudioConfig</span><span class="p">(</span>
        <span class="n">audio_encoding</span><span class="o">=</span><span class="n">texttospeech</span><span class="p">.</span><span class="n">AudioEncoding</span><span class="p">.</span><span class="n">LINEAR16</span>
    <span class="p">)</span>

    <span class="c1"># Perform the text-to-speech request on the text input with the selected
</span>    <span class="c1"># voice parameters and audio file type
</span>    <span class="n">response</span> <span class="o">=</span> <span class="n">client</span><span class="p">.</span><span class="n">synthesize_speech</span><span class="p">(</span>
        <span class="nb">input</span><span class="o">=</span><span class="n">synthesis_input</span><span class="p">,</span> <span class="n">voice</span><span class="o">=</span><span class="n">voice</span><span class="p">,</span> <span class="n">audio_config</span><span class="o">=</span><span class="n">audio_config</span>
    <span class="p">)</span>

    <span class="c1"># The response's audio_content is binary. save it as mp3 with uuid filename
</span>    <span class="n">f</span> <span class="o">=</span> <span class="nb">str</span><span class="p">(</span><span class="n">uuid</span><span class="p">.</span><span class="n">uuid4</span><span class="p">())</span><span class="o">+</span><span class="s">".wav"</span>
    <span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="n">f</span><span class="p">,</span> <span class="s">"wb"</span><span class="p">)</span> <span class="k">as</span> <span class="n">out</span><span class="p">:</span>
        <span class="c1"># Write the response to the output file.
</span>        <span class="n">out</span><span class="p">.</span><span class="n">write</span><span class="p">(</span><span class="n">response</span><span class="p">.</span><span class="n">audio_content</span><span class="p">)</span>
        <span class="k">print</span><span class="p">(</span><span class="s">'Audio content written to file {}'</span><span class="p">.</span><span class="nb">format</span><span class="p">(</span><span class="n">f</span><span class="p">))</span>
        <span class="n">f</span><span class="o">=</span><span class="s">""</span>

<span class="k">def</span> <span class="nf">synthetizeZ_text</span><span class="p">(</span><span class="n">t</span><span class="p">):</span>
    <span class="c1">#quoted from https://ponkichi.blog/voicevox-python-curl/
</span>    <span class="n">speaker</span>     <span class="o">=</span> <span class="mi">3</span>
    <span class="n">texts</span> <span class="o">=</span> <span class="n">t</span><span class="p">.</span><span class="n">split</span><span class="p">(</span><span class="s">'。'</span><span class="p">)</span>
    <span class="c1"># 音声合成処理のループ
</span>    <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">text</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">texts</span><span class="p">):</span>
        <span class="c1"># 文字列が空の場合は処理しない
</span>        <span class="k">if</span> <span class="n">text</span> <span class="o">==</span> <span class="s">''</span><span class="p">:</span> <span class="k">continue</span>

        <span class="c1"># audio_query (音声合成用のクエリを作成するAPI)
</span>        <span class="n">res1</span> <span class="o">=</span> <span class="n">requests</span><span class="p">.</span><span class="n">post</span><span class="p">(</span><span class="s">'http://'</span> <span class="o">+</span> <span class="n">VOICEVOX_HOST</span> <span class="o">+</span> <span class="s">':50021/audio_query'</span><span class="p">,</span>
                            <span class="n">params</span><span class="o">=</span><span class="p">{</span><span class="s">'text'</span><span class="p">:</span> <span class="n">text</span><span class="p">,</span> <span class="s">'speaker'</span><span class="p">:</span> <span class="n">speaker</span><span class="p">})</span>
        <span class="c1"># synthesis (音声合成するAPI)
</span>        <span class="n">res2</span> <span class="o">=</span> <span class="n">requests</span><span class="p">.</span><span class="n">post</span><span class="p">(</span><span class="s">'http://'</span> <span class="o">+</span> <span class="n">VOICEVOX_HOST</span> <span class="o">+</span> <span class="s">':50021/synthesis'</span><span class="p">,</span>
                            <span class="n">params</span><span class="o">=</span><span class="p">{</span><span class="s">'speaker'</span><span class="p">:</span> <span class="n">speaker</span><span class="p">},</span>
                            <span class="n">data</span><span class="o">=</span><span class="n">json</span><span class="p">.</span><span class="n">dumps</span><span class="p">(</span><span class="n">res1</span><span class="p">.</span><span class="n">json</span><span class="p">()))</span>

        <span class="n">f</span> <span class="o">=</span> <span class="nb">str</span><span class="p">(</span><span class="n">uuid</span><span class="p">.</span><span class="n">uuid4</span><span class="p">())</span><span class="o">+</span><span class="s">".wav"</span>
        <span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="n">f</span><span class="p">,</span> <span class="s">"wb"</span><span class="p">)</span> <span class="k">as</span> <span class="n">out</span><span class="p">:</span>
            <span class="c1"># Write the response to the output file.
</span>            <span class="n">out</span><span class="p">.</span><span class="n">write</span><span class="p">(</span><span class="n">res2</span><span class="p">.</span><span class="n">content</span><span class="p">)</span>
            <span class="k">print</span><span class="p">(</span><span class="s">'Audio content written to file {}'</span><span class="p">.</span><span class="nb">format</span><span class="p">(</span><span class="n">f</span><span class="p">))</span>
            <span class="n">f</span><span class="o">=</span><span class="s">""</span>

<span class="k">def</span> <span class="nf">main</span><span class="p">():</span>
    <span class="c1"># See http://g.co/cloud/speech/docs/languages
</span>    <span class="c1"># for a list of supported languages.
</span>    <span class="c1">#language_code = "en-US"  # a BCP-47 language tag
</span>    <span class="n">language_code</span> <span class="o">=</span> <span class="s">"ja-JP"</span>

    <span class="n">client</span> <span class="o">=</span> <span class="n">speech</span><span class="p">.</span><span class="n">SpeechClient</span><span class="p">()</span>
    <span class="n">config</span> <span class="o">=</span> <span class="n">speech</span><span class="p">.</span><span class="n">RecognitionConfig</span><span class="p">(</span>
        <span class="n">encoding</span><span class="o">=</span><span class="n">speech</span><span class="p">.</span><span class="n">RecognitionConfig</span><span class="p">.</span><span class="n">AudioEncoding</span><span class="p">.</span><span class="n">LINEAR16</span><span class="p">,</span>
        <span class="n">sample_rate_hertz</span><span class="o">=</span><span class="n">RATE</span><span class="p">,</span>
        <span class="n">language_code</span><span class="o">=</span><span class="n">language_code</span><span class="p">,</span>
    <span class="p">)</span>

    <span class="n">streaming_config</span> <span class="o">=</span> <span class="n">speech</span><span class="p">.</span><span class="n">StreamingRecognitionConfig</span><span class="p">(</span>
        <span class="n">config</span><span class="o">=</span><span class="n">config</span><span class="p">,</span> <span class="n">interim_results</span><span class="o">=</span><span class="bp">True</span>
    <span class="p">)</span>

    <span class="k">with</span> <span class="n">MicrophoneStream</span><span class="p">(</span><span class="n">RATE</span><span class="p">,</span> <span class="n">CHUNK</span><span class="p">)</span> <span class="k">as</span> <span class="n">stream</span><span class="p">:</span>
        <span class="n">audio_generator</span> <span class="o">=</span> <span class="n">stream</span><span class="p">.</span><span class="n">generator</span><span class="p">()</span>
        <span class="n">requests</span> <span class="o">=</span> <span class="p">(</span>
            <span class="n">speech</span><span class="p">.</span><span class="n">StreamingRecognizeRequest</span><span class="p">(</span><span class="n">audio_content</span><span class="o">=</span><span class="n">content</span><span class="p">)</span>
            <span class="k">for</span> <span class="n">content</span> <span class="ow">in</span> <span class="n">audio_generator</span>
        <span class="p">)</span>

        <span class="n">responses</span> <span class="o">=</span> <span class="n">client</span><span class="p">.</span><span class="n">streaming_recognize</span><span class="p">(</span><span class="n">streaming_config</span><span class="p">,</span> <span class="n">requests</span><span class="p">)</span>

        <span class="c1"># Now, put the transcription responses to use.
</span>        <span class="n">listen_print_loop</span><span class="p">(</span><span class="n">responses</span><span class="p">)</span>


<span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="s">"__main__"</span><span class="p">:</span>
    <span class="n">main</span><span class="p">()</span>
</code></pre></div></div>

<h3 id="prepare-google-api-key">Prepare Google API KEY</h3>

<p>Retrieve API KEY as json file, place it in secured path and indicate that location as GOOGLE_APPLICATION_CREDENTIALS envinment variable.</p>

<p>Please follow below:
<a href="https://cloud.google.com/speech-to-text/docs/before-you-begin">https://cloud.google.com/speech-to-text/docs/before-you-begin</a></p>

<h3 id="putting-altogether">Putting altogether</h3>

<p>Putting everything into tables from terminal and just talk.</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./play.sh &amp;
~/.voicevox/VOICEVOX.AppImage &amp;
python3 voice-transcript-synthesized.py
</code></pre></div></div>]]></content><author><name>Tomohiko Araki</name></author><summary type="html"><![CDATA[VOICEVOX make to feel better, as your voice is converted to cute Zundamon. In this article I am going to show to how to do it; the result is clutterted and crappy still due to my lack of audio processing skill, but this just works.]]></summary></entry></feed>