<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
  <author>
    <name>Road</name>
  </author>
  <generator uri="https://hexo.io/">Hexo</generator>
  <id>https://blog.malu.tech/</id>
  <link href="https://blog.malu.tech/" rel="alternate"/>
  <link href="https://blog.malu.tech/atom.xml" rel="self"/>
  <rights>All rights reserved 2026, Road</rights>
  <subtitle>这里是 @大马路叫Road 的个人博客</subtitle>
  <title>Road's Blog</title>
  <updated>2026-06-01T16:00:00.000Z</updated>
  <entry>
    <author>
      <name>Road</name>
    </author>
    <category term="工具分享" scheme="https://blog.malu.tech/categories/%E5%B7%A5%E5%85%B7%E5%88%86%E4%BA%AB/"/>
    <category term="下载器" scheme="https://blog.malu.tech/categories/%E5%B7%A5%E5%85%B7%E5%88%86%E4%BA%AB/%E4%B8%8B%E8%BD%BD%E5%99%A8/"/>
    <category term="Jable.tv" scheme="https://blog.malu.tech/tags/Jable-tv/"/>
    <category term="下载器" scheme="https://blog.malu.tech/tags/%E4%B8%8B%E8%BD%BD%E5%99%A8/"/>
    <category term="视频下载" scheme="https://blog.malu.tech/tags/%E8%A7%86%E9%A2%91%E4%B8%8B%E8%BD%BD/"/>
    <category term="WebUI" scheme="https://blog.malu.tech/tags/WebUI/"/>
    <category term="Docker" scheme="https://blog.malu.tech/tags/Docker/"/>
    <category term="Python" scheme="https://blog.malu.tech/tags/Python/"/>
    <category term="Webhook" scheme="https://blog.malu.tech/tags/Webhook/"/>
    <content>
      <![CDATA[<h1 id="JableTVDownload"><a href="#JableTVDownload" class="headerlink" title="JableTVDownload"></a>JableTVDownload</h1><p>该项目 fork 自 <a href="https://github.com/hcjohn463">hcjohn463</a> 的 <a href="https://github.com/hcjohn463/JableTVDownload">JableTVDownloader</a>，在原项目基础上增加了下载代理配置、下载队列、WebUI、浏览器 UA 配置、Webhook、防机器人认证检测等功能。</p><p>另外基于Webhook功能，可以结合iOS系统的快捷指令，实现手机触发下载任务的功能。</p><hr><h2 id="🚀-快速开始（推荐）"><a href="#🚀-快速开始（推荐）" class="headerlink" title="🚀 快速开始（推荐）"></a>🚀 快速开始（推荐）</h2><p>使用 Docker Compose 一键启动完整的 Web UI + 下载器服务，适合家庭 NAS 或服务器部署。</p><h3 id="前置要求"><a href="#前置要求" class="headerlink" title="前置要求"></a>前置要求</h3><ul><li>安装 <a href="https://docs.docker.com/engine/install/">Docker Engine</a></li><li>安装 <a href="https://docs.docker.com/compose/install/">Docker Compose</a></li></ul><h3 id="启动方式"><a href="#启动方式" class="headerlink" title="启动方式"></a>启动方式</h3><figure class="highlight bash"><table><tr><td class="code"><pre><span class="line"><span class="comment"># 1. 克隆仓库</span></span><br><span class="line">git <span class="built_in">clone</span> https://github.com/Road-tech/JableTVDownload.git</span><br><span class="line"></span><br><span class="line"><span class="comment"># 2. 进入项目目录</span></span><br><span class="line"><span class="built_in">cd</span> JableTVDownload</span><br><span class="line"></span><br><span class="line"><span class="comment"># 3. 复制完整配置文件</span></span><br><span class="line"><span class="built_in">cp</span> docker-compose.full.yml docker-compose.yml</span><br><span class="line"></span><br><span class="line"><span class="comment"># 4. 编辑配置文件（可选，根据需要修改端口、下载目录等，具体说明见下文）</span></span><br><span class="line">vi docker-compose.yml</span><br><span class="line"></span><br><span class="line"><span class="comment"># 5. 启动所有服务</span></span><br><span class="line">docker compose up -d</span><br></pre></td></tr></table></figure><h3 id="🐳-docker-compose配置说明（docker-compose-full-yml）"><a href="#🐳-docker-compose配置说明（docker-compose-full-yml）" class="headerlink" title="🐳 docker-compose配置说明（docker-compose.full.yml）"></a>🐳 docker-compose配置说明（docker-compose.full.yml）</h3><figure class="highlight yaml"><table><tr><td class="code"><pre><span class="line"><span class="attr">services:</span></span><br><span class="line">  <span class="attr">webui:</span></span><br><span class="line">    <span class="attr">ports:</span></span><br><span class="line">      <span class="bullet">-</span> <span class="string">&quot;8080:8080&quot;</span>  <span class="comment"># Web UI 端口，可修改前面的数字</span></span><br><span class="line">    <span class="attr">environment:</span></span><br><span class="line">      <span class="bullet">-</span> <span class="string">DOWNLOADER_API=http://downloader:5000</span>  <span class="comment"># 下载器 API 地址</span></span><br><span class="line">    <span class="attr">volumes:</span></span><br><span class="line">      <span class="bullet">-</span> <span class="string">./logs:/logs</span>  <span class="comment"># 日志目录（任务记录 tasks.json）</span></span><br><span class="line">  </span><br><span class="line">  <span class="attr">downloader:</span></span><br><span class="line">    <span class="attr">ports:</span></span><br><span class="line">      <span class="bullet">-</span> <span class="string">&quot;5000:5000&quot;</span>  <span class="comment"># API 端口，可修改前面的数字</span></span><br><span class="line">    <span class="attr">environment:</span></span><br><span class="line">      <span class="bullet">-</span> <span class="string">SERVER=true</span>           <span class="comment"># 启用 Webhook 服务器</span></span><br><span class="line">      <span class="bullet">-</span> <span class="string">HOST=0.0.0.0</span>         <span class="comment"># 监听地址</span></span><br><span class="line">      <span class="bullet">-</span> <span class="string">PORT=5000</span>            <span class="comment"># 服务端口</span></span><br><span class="line">      <span class="bullet">-</span> <span class="string">LOG_DIR=/logs</span>        <span class="comment"># 日志目录</span></span><br><span class="line">    <span class="attr">volumes:</span></span><br><span class="line">      <span class="bullet">-</span> <span class="string">./downloads:/downloads</span>                 <span class="comment"># 下载目录</span></span><br><span class="line">      <span class="bullet">-</span> <span class="string">./downloader/config.json:/app/config.json</span>  <span class="comment"># 下载器配置文件，具体配置说明见下文</span></span><br><span class="line">      <span class="bullet">-</span> <span class="string">./logs:/logs</span>         <span class="comment"># 日志目录</span></span><br><span class="line"></span><br><span class="line"><span class="attr">networks:</span></span><br><span class="line">  <span class="attr">jable-network:</span></span><br><span class="line">    <span class="attr">driver:</span> <span class="string">bridge</span></span><br><span class="line"></span><br></pre></td></tr></table></figure><h3 id="📋-下载器配置文件-config-json"><a href="#📋-下载器配置文件-config-json" class="headerlink" title="📋 下载器配置文件 (config.json)"></a>📋 下载器配置文件 (config.json)</h3><p>可以看到上面的docker-compose.full.yml挂载了config.json，这个JSON配置文件管理下载器的常用设置，无需每次输入参数。当然你可以不用修改config.json，直接通过WebUI进行配置。</p><figure class="highlight json"><table><tr><td class="code"><pre><span class="line"><span class="punctuation">&#123;</span></span><br><span class="line">    <span class="attr">&quot;proxy&quot;</span><span class="punctuation">:</span> <span class="punctuation">&#123;</span></span><br><span class="line">        <span class="attr">&quot;enabled&quot;</span><span class="punctuation">:</span> <span class="literal"><span class="keyword">false</span></span><span class="punctuation">,</span>  <span class="comment">// 是否启用代理，默认 false</span></span><br><span class="line">        <span class="attr">&quot;url&quot;</span><span class="punctuation">:</span> <span class="string">&quot;&quot;</span>          <span class="comment">// 代理服务器地址，如: &quot;http://proxy:8080&quot;</span></span><br><span class="line">    <span class="punctuation">&#125;</span><span class="punctuation">,</span></span><br><span class="line">    <span class="attr">&quot;browser&quot;</span><span class="punctuation">:</span> <span class="punctuation">&#123;</span></span><br><span class="line">        <span class="attr">&quot;user_agent&quot;</span><span class="punctuation">:</span> <span class="string">&quot;Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36&quot;</span><span class="punctuation">,</span>  <span class="comment">// 浏览器 User Agent</span></span><br><span class="line">        <span class="attr">&quot;headless&quot;</span><span class="punctuation">:</span> <span class="literal"><span class="keyword">true</span></span><span class="punctuation">,</span>       <span class="comment">// 是否使用无头模式（不显示浏览器窗口），默认 true</span></span><br><span class="line">        <span class="attr">&quot;disable_images&quot;</span><span class="punctuation">:</span> <span class="literal"><span class="keyword">false</span></span><span class="punctuation">,</span> <span class="comment">// 是否禁用图片加载（加快速度），默认 false</span></span><br><span class="line">        <span class="attr">&quot;page_load_timeout&quot;</span><span class="punctuation">:</span> <span class="number">30</span><span class="punctuation">,</span> <span class="comment">// 页面加载超时时间（秒），默认 30</span></span><br><span class="line">        <span class="attr">&quot;implicit_wait&quot;</span><span class="punctuation">:</span> <span class="number">10</span>     <span class="comment">// 隐式等待时间（秒），默认 10</span></span><br><span class="line">    <span class="punctuation">&#125;</span><span class="punctuation">,</span></span><br><span class="line">    <span class="attr">&quot;download&quot;</span><span class="punctuation">:</span> <span class="punctuation">&#123;</span></span><br><span class="line">        <span class="attr">&quot;cover&quot;</span><span class="punctuation">:</span> <span class="literal"><span class="keyword">true</span></span><span class="punctuation">,</span>  <span class="comment">// 是否下载视频封面，默认 true</span></span><br><span class="line">        <span class="attr">&quot;encode&quot;</span><span class="punctuation">:</span> <span class="literal"><span class="keyword">true</span></span><span class="punctuation">,</span> <span class="comment">// 是否使用 FFmpeg 转码优化，默认 true</span></span><br><span class="line">        <span class="attr">&quot;quality&quot;</span><span class="punctuation">:</span> <span class="number">1</span>    <span class="comment">// 转码质量 (1=最快, 2=适中, 3=最佳)，默认 1</span></span><br><span class="line">    <span class="punctuation">&#125;</span></span><br><span class="line"><span class="punctuation">&#125;</span></span><br></pre></td></tr></table></figure><h3 id="🌐-浏览器访问WebUI"><a href="#🌐-浏览器访问WebUI" class="headerlink" title="🌐 浏览器访问WebUI"></a>🌐 浏览器访问WebUI</h3><p>启动成功后，通过浏览器访问WebUI：</p><ul><li><a href="http://localhost:8080/">http://localhost:8080</a></li></ul><hr><h2 id="📂-项目结构"><a href="#📂-项目结构" class="headerlink" title="📂 项目结构"></a>📂 项目结构</h2><figure class="highlight nix"><table><tr><td class="code"><pre><span class="line">JableTVDownload<span class="symbol">/</span></span><br><span class="line">├── docker-compose.full.yml  <span class="comment"># Docker Compose 完整配置模板</span></span><br><span class="line">├── README.md               <span class="comment"># 项目说明文档</span></span><br><span class="line">├── start.sh                <span class="comment"># Linux/Mac 启动脚本</span></span><br><span class="line">├── start.bat               <span class="comment"># Windows 启动脚本</span></span><br><span class="line">├── downloader<span class="symbol">/</span>             <span class="comment"># 下载器核心代码</span></span><br><span class="line">│   ├── Dockerfile          <span class="comment"># Downloader 镜像构建文件</span></span><br><span class="line">│   ├── config.json         <span class="comment"># 下载器配置文件</span></span><br><span class="line">│   ├── main.py             <span class="comment"># 主入口</span></span><br><span class="line">│   ├── crawler.py          <span class="comment"># 网页爬虫</span></span><br><span class="line">│   ├── download.py         <span class="comment"># 下载模块</span></span><br><span class="line">│   ├── merge.py            <span class="comment"># 视频合并</span></span><br><span class="line">│   ├── encode.py           <span class="comment"># 转码模块</span></span><br><span class="line">│   ├── cover.py            <span class="comment"># 封面下载</span></span><br><span class="line">│   ├── webhook_server.py   <span class="comment"># Webhook API 服务</span></span><br><span class="line">│   └── requirements.txt    <span class="comment"># Python 依赖</span></span><br><span class="line">├── webui<span class="symbol">/</span>                  <span class="comment"># Web UI 前端代码</span></span><br><span class="line">│   ├── Dockerfile.webui    <span class="comment"># Web UI 镜像构建文件</span></span><br><span class="line">│   ├── web_ui.py           <span class="comment"># Web UI 主程序</span></span><br><span class="line">│   ├── templates<span class="symbol">/</span>          <span class="comment"># HTML 模板</span></span><br><span class="line">│   └── requirements_webui.txt</span><br><span class="line">└── img<span class="symbol">/</span>                    <span class="comment"># 文档图片资源</span></span><br></pre></td></tr></table></figure><hr><h2 id="项目功能"><a href="#项目功能" class="headerlink" title="项目功能"></a>项目功能</h2><h3 id="🎨-WebUI"><a href="#🎨-WebUI" class="headerlink" title="🎨 WebUI"></a>🎨 WebUI</h3><p>提供直观的网页界面（<a href="https://htmlpreview.github.io/?https://github.com/Road-tech/JableTVDownload/blob/main/webui/templates/index.html">预览界面</a>），无需编写代码或使用终端即可使用所有功能，功能特性如下：</p><h4 id="1-添加下载任务"><a href="#1-添加下载任务" class="headerlink" title="1. 添加下载任务"></a>1. 添加下载任务</h4><ul><li>支持完整 URL（如：<code>https://jable.tv/videos/snxx-XXX/</code>）</li><li>支持仅输入番号（如：<code>snxx-XXX</code>），系统自动补全</li><li>支持批量添加（每行输入一个）</li><li>任务自动加入下载队列</li></ul><h4 id="2-下载队列管理"><a href="#2-下载队列管理" class="headerlink" title="2. 下载队列管理"></a>2. 下载队列管理</h4><ul><li>任务按添加顺序排队</li><li>默认同时下载 1 个任务</li><li>可配置同时下载数量（1-8）</li></ul><h4 id="3-实时监控"><a href="#3-实时监控" class="headerlink" title="3. 实时监控"></a>3. 实时监控</h4><ul><li>下载任务列表</li><li>实时日志输出</li><li>下载进度条显示</li></ul><h4 id="4-配置管理"><a href="#4-配置管理" class="headerlink" title="4. 配置管理"></a>4. 配置管理</h4><ul><li>代理设置（启用&#x2F;停用、代理地址）</li><li>浏览器设置（无头模式、禁用图片加载、自定义 UA）</li><li>下载设置（封面下载、转码、转码质量）</li></ul><hr><h3 id="🌐-Webhook-API-服务"><a href="#🌐-Webhook-API-服务" class="headerlink" title="🌐 Webhook API 服务"></a>🌐 Webhook API 服务</h3><p>支持通过 HTTP API 调用下载任务，方便与其他系统集成。</p><h4 id="API-端点"><a href="#API-端点" class="headerlink" title="API 端点"></a>API 端点</h4><table><thead><tr><th align="left">端点</th><th align="left">方法</th><th align="left">说明</th></tr></thead><tbody><tr><td align="left"><code>/health</code></td><td align="left">GET</td><td align="left">健康检查</td></tr><tr><td align="left"><code>/api/download</code></td><td align="left">POST</td><td align="left">添加下载任务</td></tr><tr><td align="left"><code>/api/download/batch</code></td><td align="left">POST</td><td align="left">批量添加下载任务</td></tr><tr><td align="left"><code>/api/config</code></td><td align="left">GET</td><td align="left">获取当前配置</td></tr><tr><td align="left"><code>/api/config</code></td><td align="left">PUT&#x2F;POST</td><td align="left">更新配置</td></tr><tr><td align="left"><code>/api/tasks</code></td><td align="left">GET</td><td align="left">获取下载任务列表</td></tr><tr><td align="left"><code>/api/tasks/&lt;task_id&gt;</code></td><td align="left">DELETE</td><td align="left">删除指定任务</td></tr><tr><td align="left"><code>/api/tasks/&lt;task_id&gt;/stop</code></td><td align="left">POST</td><td align="left">停止正在下载的任务</td></tr><tr><td align="left"><code>/api/tasks</code></td><td align="left">DELETE</td><td align="left">清除已完成的任务</td></tr><tr><td align="left"><code>/api/progress</code></td><td align="left">GET</td><td align="left">获取当前下载进度</td></tr><tr><td align="left"><code>/api/queue/status</code></td><td align="left">GET</td><td align="left">获取队列状态</td></tr></tbody></table><h4 id="API-使用示例"><a href="#API-使用示例" class="headerlink" title="API 使用示例"></a>API 使用示例</h4><h5 id="1-添加下载任务-1"><a href="#1-添加下载任务-1" class="headerlink" title="1. 添加下载任务"></a>1. 添加下载任务</h5><figure class="highlight bash"><table><tr><td class="code"><pre><span class="line">curl -X POST http://localhost:5000/api/download \</span><br><span class="line">  -H <span class="string">&quot;Content-Type: application/json&quot;</span> \</span><br><span class="line">  -d <span class="string">&#x27;&#123;&quot;url&quot;: &quot;https://jable.tv/videos/xxx/&quot;&#125;&#x27;</span></span><br></pre></td></tr></table></figure><p>可选参数：</p><figure class="highlight json"><table><tr><td class="code"><pre><span class="line"><span class="punctuation">&#123;</span></span><br><span class="line">  <span class="attr">&quot;url&quot;</span><span class="punctuation">:</span> <span class="string">&quot;https://jable.tv/videos/xxx/&quot;</span><span class="punctuation">,</span></span><br><span class="line">  <span class="attr">&quot;cover&quot;</span><span class="punctuation">:</span> <span class="literal"><span class="keyword">true</span></span><span class="punctuation">,</span></span><br><span class="line">  <span class="attr">&quot;encode&quot;</span><span class="punctuation">:</span> <span class="literal"><span class="keyword">true</span></span><span class="punctuation">,</span></span><br><span class="line">  <span class="attr">&quot;quality&quot;</span><span class="punctuation">:</span> <span class="number">1</span><span class="punctuation">,</span></span><br><span class="line">  <span class="attr">&quot;proxy&quot;</span><span class="punctuation">:</span> <span class="string">&quot;http://proxy.example.com:8080&quot;</span></span><br><span class="line"><span class="punctuation">&#125;</span></span><br></pre></td></tr></table></figure><h5 id="2-获取当前配置"><a href="#2-获取当前配置" class="headerlink" title="2. 获取当前配置"></a>2. 获取当前配置</h5><figure class="highlight bash"><table><tr><td class="code"><pre><span class="line">curl http://localhost:5000/api/config</span><br></pre></td></tr></table></figure><h5 id="3-更新配置"><a href="#3-更新配置" class="headerlink" title="3. 更新配置"></a>3. 更新配置</h5><figure class="highlight bash"><table><tr><td class="code"><pre><span class="line">curl -X PUT http://localhost:5000/api/config \</span><br><span class="line">  -H <span class="string">&quot;Content-Type: application/json&quot;</span> \</span><br><span class="line">  -d <span class="string">&#x27;&#123;</span></span><br><span class="line"><span class="string">    &quot;proxy&quot;: &#123;</span></span><br><span class="line"><span class="string">      &quot;enabled&quot;: true,</span></span><br><span class="line"><span class="string">      &quot;url&quot;: &quot;http://proxy.example.com:8080&quot;</span></span><br><span class="line"><span class="string">    &#125;,</span></span><br><span class="line"><span class="string">    &quot;download&quot;: &#123;</span></span><br><span class="line"><span class="string">      &quot;cover&quot;: false,</span></span><br><span class="line"><span class="string">      &quot;quality&quot;: 2</span></span><br><span class="line"><span class="string">    &#125;</span></span><br><span class="line"><span class="string">  &#125;&#x27;</span></span><br></pre></td></tr></table></figure><h5 id="4-查看下载任务"><a href="#4-查看下载任务" class="headerlink" title="4. 查看下载任务"></a>4. 查看下载任务</h5><figure class="highlight bash"><table><tr><td class="code"><pre><span class="line"><span class="comment"># 查看所有任务</span></span><br><span class="line">curl http://localhost:5000/api/tasks</span><br><span class="line"></span><br><span class="line"><span class="comment"># 筛选任务 (all/pending/downloading/completed/error)</span></span><br><span class="line">curl http://localhost:5000/api/tasks?status=downloading</span><br></pre></td></tr></table></figure><h5 id="5-停止下载任务"><a href="#5-停止下载任务" class="headerlink" title="5. 停止下载任务"></a>5. 停止下载任务</h5><figure class="highlight bash"><table><tr><td class="code"><pre><span class="line">curl -X POST http://localhost:5000/api/tasks/1/stop</span><br></pre></td></tr></table></figure><h5 id="6-删除指定任务"><a href="#6-删除指定任务" class="headerlink" title="6. 删除指定任务"></a>6. 删除指定任务</h5><figure class="highlight bash"><table><tr><td class="code"><pre><span class="line">curl -X DELETE http://localhost:5000/api/tasks/1</span><br></pre></td></tr></table></figure><h5 id="7-清除已完成的任务"><a href="#7-清除已完成的任务" class="headerlink" title="7. 清除已完成的任务"></a>7. 清除已完成的任务</h5><figure class="highlight bash"><table><tr><td class="code"><pre><span class="line"><span class="comment"># 清除已完成和错误的任务</span></span><br><span class="line">curl -X DELETE http://localhost:5000/api/tasks?filter=completed</span><br><span class="line"></span><br><span class="line"><span class="comment"># 清除所有任务</span></span><br><span class="line">curl -X DELETE http://localhost:5000/api/tasks?filter=all</span><br></pre></td></tr></table></figure><h5 id="8-获取下载进度"><a href="#8-获取下载进度" class="headerlink" title="8. 获取下载进度"></a>8. 获取下载进度</h5><figure class="highlight bash"><table><tr><td class="code"><pre><span class="line"><span class="comment"># 获取所有任务进度</span></span><br><span class="line">curl http://localhost:5000/api/progress</span><br><span class="line"></span><br><span class="line"><span class="comment"># 获取指定任务进度</span></span><br><span class="line">curl http://localhost:5000/api/progress?task_id=1</span><br></pre></td></tr></table></figure><hr><h3 id="🐳-通过Docker的容器直接下载"><a href="#🐳-通过Docker的容器直接下载" class="headerlink" title="🐳 通过Docker的容器直接下载"></a>🐳 通过Docker的容器直接下载</h3><p>如果不喜欢WebUI的方式，也可以直接创建容器，把视频下载到指定目录。</p><h4 id="交互模式"><a href="#交互模式" class="headerlink" title="交互模式"></a>交互模式</h4><figure class="highlight bash"><table><tr><td class="code"><pre><span class="line"><span class="comment"># 1. 构建镜像</span></span><br><span class="line">docker build -t jable-downloader .</span><br><span class="line"></span><br><span class="line"><span class="comment"># 2. 运行（交互模式，下载视频存至本地 downloads 文件夹）</span></span><br><span class="line">docker run -it -v D:\downloads:/downloads jable-downloader</span><br></pre></td></tr></table></figure><h4 id="基于配置文件直接下载视频"><a href="#基于配置文件直接下载视频" class="headerlink" title="基于配置文件直接下载视频"></a>基于配置文件直接下载视频</h4><p>将本地的 <code>config.json</code> 挂载到容器内：</p><figure class="highlight bash"><table><tr><td class="code"><pre><span class="line">docker run -it \</span><br><span class="line">  -v ./downloads:/downloads \</span><br><span class="line">  -v ./config.json:/app/config.json \</span><br><span class="line">  -e URL=<span class="string">&quot;https://jable.tv/videos/xxx/&quot;</span> \</span><br><span class="line">  jable-downloader</span><br></pre></td></tr></table></figure><h4 id="基于环境变量配置下载视频"><a href="#基于环境变量配置下载视频" class="headerlink" title="基于环境变量配置下载视频"></a>基于环境变量配置下载视频</h4><figure class="highlight bash"><table><tr><td class="code"><pre><span class="line">docker run -it \</span><br><span class="line">  -v ./downloads:/downloads \</span><br><span class="line">  -v ./config.json:/app/config.json \</span><br><span class="line">  -e URL=<span class="string">&quot;https://jable.tv/videos/xxx/&quot;</span> \</span><br><span class="line">  -e PROXY=<span class="string">&quot;http://proxy.example.com:8080&quot;</span> \</span><br><span class="line">  -e ENABLE_PROXY=<span class="string">&quot;true&quot;</span> \</span><br><span class="line">  -e COVER=<span class="string">&quot;False&quot;</span> \</span><br><span class="line">  -e QUALITY=<span class="string">&quot;2&quot;</span> \</span><br><span class="line">  jable-downloader</span><br></pre></td></tr></table></figure><h5 id="环境变量列表"><a href="#环境变量列表" class="headerlink" title="环境变量列表"></a>环境变量列表</h5><table><thead><tr><th align="left">环境变量</th><th align="left">说明</th></tr></thead><tbody><tr><td align="left"><code>SERVER</code></td><td align="left">启动 Webhook 服务器 (true)</td></tr><tr><td align="left"><code>HOST</code></td><td align="left">服务器监听地址 (默认: 0.0.0.0)</td></tr><tr><td align="left"><code>PORT</code></td><td align="left">服务器端口 (默认: 5000)</td></tr><tr><td align="left"><code>URL</code></td><td align="left">视频网址</td></tr><tr><td align="left"><code>RANDOM</code></td><td align="left">下载随机热门视频 (true&#x2F;false)</td></tr><tr><td align="left"><code>ALL_URLS</code></td><td align="left">演员页网址，下载所有视频</td></tr><tr><td align="left"><code>CONFIG</code></td><td align="left">自定义配置文件路径（容器内）</td></tr><tr><td align="left"><code>PROXY</code></td><td align="left">代理地址</td></tr><tr><td align="left"><code>ENABLE_PROXY</code></td><td align="left">启用代理 (true)</td></tr><tr><td align="left"><code>DISABLE_PROXY</code></td><td align="left">停用代理 (true)</td></tr><tr><td align="left"><code>COVER</code></td><td align="left">是否下载封面 (true&#x2F;false)</td></tr><tr><td align="left"><code>ENCODE</code></td><td align="left">是否转码 (true&#x2F;false)</td></tr><tr><td align="left"><code>QUALITY</code></td><td align="left">转码质量 (1&#x2F;2&#x2F;3)</td></tr></tbody></table><hr><h3 id="直接运行"><a href="#直接运行" class="headerlink" title="直接运行"></a>直接运行</h3><p>直接运行程序、不启动docker容器</p><h4 id="⌨️-命令行参数"><a href="#⌨️-命令行参数" class="headerlink" title="⌨️ 命令行参数"></a>⌨️ 命令行参数</h4><figure class="highlight bash"><table><tr><td class="code"><pre><span class="line">python main.py -h</span><br><span class="line"></span><br><span class="line"><span class="comment"># Webhook 服务器模式</span></span><br><span class="line">python main.py --server                    <span class="comment"># 启动 Webhook 服务器</span></span><br><span class="line">python main.py --server --host 0.0.0.0 --port 5000  <span class="comment"># 自定义监听地址和端口</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 下载模式</span></span><br><span class="line">python main.py --random True       <span class="comment"># 下载随机热门视频</span></span><br><span class="line">python main.py --url &lt;网址&gt;         <span class="comment"># 直接指定 URL</span></span><br><span class="line">python main.py --all-urls &lt;演员页&gt;  <span class="comment"># 下载演员所有视频</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 配置文件相关</span></span><br><span class="line">python main.py --config my_config.json  <span class="comment"># 使用自定义配置文件</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 代理相关</span></span><br><span class="line">python main.py --enable-proxy                          <span class="comment"># 启用代理</span></span><br><span class="line">python main.py --disable-proxy                         <span class="comment"># 停用代理</span></span><br><span class="line">python main.py --proxy <span class="string">&quot;http://proxy.example.com:8080&quot;</span> <span class="comment"># 设置代理地址</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 下载选项</span></span><br><span class="line">python main.py --cover False  <span class="comment"># 不下载封面</span></span><br><span class="line">python main.py --encode False <span class="comment"># 不进行转码</span></span><br><span class="line">python main.py --quality 2    <span class="comment"># 设置转码质量 (1/2/3)</span></span><br></pre></td></tr></table></figure><h4 id="参数优先级"><a href="#参数优先级" class="headerlink" title="参数优先级"></a>参数优先级</h4><p>命令行参数优先级高于配置文件，例如：</p><figure class="highlight bash"><table><tr><td class="code"><pre><span class="line"><span class="comment"># 使用配置文件的代理设置，但关闭封面下载</span></span><br><span class="line">python main.py --url &lt;网址&gt; --cover False</span><br></pre></td></tr></table></figure><hr><h2 id="🛠️-维护命令"><a href="#🛠️-维护命令" class="headerlink" title="🛠️ 维护命令"></a>🛠️ 维护命令</h2><figure class="highlight bash"><table><tr><td class="code"><pre><span class="line"><span class="comment"># 停止所有服务</span></span><br><span class="line">docker compose down</span><br><span class="line"></span><br><span class="line"><span class="comment"># 重新构建镜像</span></span><br><span class="line">docker compose build --no-cache</span><br><span class="line"></span><br><span class="line"><span class="comment"># 重启服务</span></span><br><span class="line">docker compose restart</span><br><span class="line"></span><br><span class="line"><span class="comment"># 查看实时日志</span></span><br><span class="line">docker compose logs -f webui</span><br><span class="line">docker compose logs -f downloader</span><br><span class="line"></span><br><span class="line"><span class="comment"># 清理未使用的镜像</span></span><br><span class="line">docker image prune -f</span><br></pre></td></tr></table></figure><hr><h2 id="📜-更新日志"><a href="#📜-更新日志" class="headerlink" title="📜 更新日志"></a>📜 更新日志</h2><table><thead><tr><th align="left">版本</th><th align="left">日期</th><th align="left">内容</th></tr></thead><tbody><tr><td align="left"><strong>v3.0</strong></td><td align="left">2026&#x2F;06&#x2F;01</td><td align="left">🌟 新增完整 Web UI 图形化界面</td></tr><tr><td align="left"></td><td align="left"></td><td align="left">🎨 支持直接输入番号自动补全 URL</td></tr><tr><td align="left"></td><td align="left"></td><td align="left">📊 实时显示任务进度和日志</td></tr><tr><td align="left"></td><td align="left"></td><td align="left">⚙️ Web UI 配置管理功能</td></tr><tr><td align="left"><strong>v2.3</strong></td><td align="left">2026&#x2F;05&#x2F;30</td><td align="left">🛡️ 新增浏览器配置项（UA、超时等）</td></tr><tr><td align="left"></td><td align="left"></td><td align="left">🚫 新增机器人验证检测与提示功能</td></tr><tr><td align="left"><strong>v2.2</strong></td><td align="left">2026&#x2F;05&#x2F;30</td><td align="left">🌐 新增 Webhook API 服务</td></tr><tr><td align="left"></td><td align="left"></td><td align="left">📡 支持 HTTP API 新建下载任务、更新设置</td></tr><tr><td align="left"><strong>v2.1</strong></td><td align="left">2026&#x2F;05&#x2F;30</td><td align="left">⚙️ 新增 <code>config.json</code> 配置文件系统</td></tr><tr><td align="left"></td><td align="left"></td><td align="left">🚀 支持通过配置文件管理代理、封面下载、转码等选项</td></tr><tr><td align="left"><strong>v2.0</strong></td><td align="left">2026&#x2F;03&#x2F;15</td><td align="left">🐳 支持 Docker 容器化部署</td></tr><tr><td align="left"></td><td align="left"></td><td align="left">☸️ K8s（Job&#x2F;PVC&#x2F;ConfigMap）支持</td></tr><tr><td align="left"></td><td align="left"></td><td align="left">📊 下载与合成加入 <code>tqdm</code> 实时进度条</td></tr><tr><td align="left"></td><td align="left"></td><td align="left">🚀 优化合成与转码速度</td></tr><tr><td align="left"><strong>v1.11</strong></td><td align="left">2023&#x2F;04&#x2F;19</td><td align="left">🦕 新增 ffmpeg 自动转码</td></tr><tr><td align="left"><strong>v1.10</strong></td><td align="left">2023&#x2F;04&#x2F;19</td><td align="left">🏹 兼容 Ubuntu Server</td></tr><tr><td align="left"><strong>v1.9</strong></td><td align="left">2023&#x2F;04&#x2F;15</td><td align="left">🦅 下载演员所有相关视频</td></tr><tr><td align="left"><strong>v1.8</strong></td><td align="left">2022&#x2F;01&#x2F;25</td><td align="left">🚗 下载结束后自动抓取封面</td></tr><tr><td align="left"><strong>v1.7</strong></td><td align="left">2021&#x2F;06&#x2F;04</td><td align="left">🐶 更改 m3u8 获取方法（正则表达式）</td></tr><tr><td align="left"><strong>v1.6</strong></td><td align="left">2021&#x2F;05&#x2F;28</td><td align="left">🌏 支持 Unix 系统（Mac、Linux 等）</td></tr><tr><td align="left"><strong>v1.5</strong></td><td align="left">2021&#x2F;05&#x2F;27</td><td align="left">🍎 更新爬虫网页方法</td></tr><tr><td align="left"><strong>v1.4</strong></td><td align="left">2021&#x2F;05&#x2F;20</td><td align="left">🌳 修改编码问题</td></tr><tr><td align="left"><strong>v1.3</strong></td><td align="left">2021&#x2F;05&#x2F;06</td><td align="left">🌈 增加下载进度提示、修改 Crypto 问题</td></tr><tr><td align="left"><strong>v1.2</strong></td><td align="left">2021&#x2F;05&#x2F;05</td><td align="left">⭐ 更新稳定版本</td></tr></tbody></table>]]>
    </content>
    <id>https://blog.malu.tech/JableTVDownload/</id>
    <link href="https://blog.malu.tech/JableTVDownload/"/>
    <published>2026-06-01T16:00:00.000Z</published>
    <summary>一个基于python的Jable.tv 视频下载工具。工具在别人的项目基础上进行升级，添加WebUI 图形界面、下载队列管理、代理配置、Webhook API、浏览器 UA 配置等功能，可通过 Docker Compose 一键部署。</summary>
    <title>小工具分享：JableTVDownload下载器</title>
    <updated>2026-06-01T16:00:00.000Z</updated>
  </entry>
  <entry>
    <author>
      <name>Road</name>
    </author>
    <category term="Python" scheme="https://blog.malu.tech/categories/Python/"/>
    <category term="Django" scheme="https://blog.malu.tech/categories/Python/Django/"/>
    <category term="数据分析" scheme="https://blog.malu.tech/categories/Python/Django/%E6%95%B0%E6%8D%AE%E5%88%86%E6%9E%90/"/>
    <category term="Python" scheme="https://blog.malu.tech/tags/Python/"/>
    <category term="Django" scheme="https://blog.malu.tech/tags/Django/"/>
    <category term="SQL" scheme="https://blog.malu.tech/tags/SQL/"/>
    <category term="Pandas" scheme="https://blog.malu.tech/tags/Pandas/"/>
    <category term="Pyecharts" scheme="https://blog.malu.tech/tags/Pyecharts/"/>
    <category term="数据分析平台" scheme="https://blog.malu.tech/tags/%E6%95%B0%E6%8D%AE%E5%88%86%E6%9E%90%E5%B9%B3%E5%8F%B0/"/>
    <content>
      <![CDATA[<blockquote><p><strong>文章来源</strong>：<a href="https://zhuanlan.zhihu.com/p/142490087">Python Django+SQL+Pandas+Pyecharts自建在线数据分析平台（一）</a><br><strong>作者</strong>：<a href="https://www.zhihu.com/people/chen-cheng-76-40">ccpic</a><br><img src="/images/python-djangosqlpand/v2-f5e0025b96485cb75f9c8a957231c941_1440w.webp"><br><img src="/images/python-djangosqlpand/v2-12c3b0e65b39e9739b54f3630b2b6a1c_1440w.webp"><br><img src="/images/python-djangosqlpand/v2-1c6b3c651e2b4e11eeb014fe5fac8696_1440w.webp"><br><strong>感谢</strong>：感谢作者 ccpic 分享的优质内容，本网页主要用于学习知识的存档备份，欢迎点击原网页支持作者。</p></blockquote><p>今天下定决心给自己挖一个超级大坑，因为这个主题涉及面太广了，篇幅也会很长。但不知道怎么了还是有了挖坑的冲动。可能随着工作经验的累积想法也出现了变化，冠冕堂皇的说法就是内心其实经历了一个从使命必达到开拓创新再到传道授业的过程。</p><p><em>笔者供职于一家上市药企做市场研究工作，没有计算机背景，编程纯属自学。平日的工作内容也很繁杂，程序化分析大约只占日常工作的20%。我把它看成自动化办公，提升分析效率与质量的手段，而可能穷极一生也不会像一些一手程序员一样领会到这些代码背后的本质。因此，本篇教程将在程序编写上没那专业（希望倒不至于漏洞百出），但相对贴近业务应用与解决实际问题，同时可能更适合传统行业非IT背景的专业人员。</em></p><h2 id="（一）-需求分析-技术实现"><a href="#（一）-需求分析-技术实现" class="headerlink" title="（一）- 需求分析&amp;技术实现"></a>（一）- 需求分析&amp;技术实现</h2><p>（二）初步搭建Django环境</p><p>（三）页面布局&amp;Django模板</p><p>（四）SQL+Pandas初步处理数据</p><p>（五）前端表单交互</p><p>（六）Ajax异步传参与加载</p><p>（七）前端数据格式的处理</p><p>（八）DataTables接管前端表格</p><p>（九）Pyecharts实现交互图表</p><p>（十）静态图表的展示</p><p>（十一）“导出数据至Excel”功能</p><p>（十二）添加和配置缓存</p><p>（十三）用户登录系统</p><p>（十四）部署Django至生产环境</p><p>既然是建分析平台，在一切开始之前不妨也分析分析自己：</p><ul><li><strong>我们的数据分析平台需要满足哪些核心需求？</strong></li><li><strong>为了满足这些需求我们至少需要建设哪些核心功能？</strong></li><li><strong>这些核心功能考虑效率和质量的最优解决方案是什么？</strong></li></ul><p>核心需求的问题其实就是我们对比现状最期待哪些地方有所提升，对于数据分析平台提升一般是指两方面，或提升效率，或提升用户体验。</p><p>想象一个传统行业，在没有程序化分析裸奔的时候，一般的数据分析主力是Excel拉数据透视表+简单的SQL。然后发现没有图表和一些Calculated Metrics，自己翻过来倒过去粘数写公式太累还容易出错，发展了数据透视图和字段公式。需要一些交互功能，再加入切片器和更复杂的公式嵌套。</p><p>一般到这算是跨过初级阶段的坎，但还是脱离不开万能的Excel的范畴，其实对多数人也够用了。之后再想进步的人开始折腾VBA+ADO。VBA极大地丰富了数据查询和数据展示的灵活性，并提升了自动化效率；ADO形成了在线数据平台的雏形，允许以一个Excel VBA的壳访问远程服务调数。这个解决方案可以说基本实现了程序化分析的初级阶段，核心问题还是无法脱离Excel的展示框架和VB的运行效率瓶颈。<br><img src="/images/python-djangosqlpand/v2-f5e9ff46471a6252c15b60786d84a981_1440w.webp"></p><p>之前在上一家MNC工作时自制的VBA+ADO数据分析工具，离职后无法访问数据服务器只剩个空壳，能看到左边是控件操作区域，右边是展示区域，由若干个工作表提供不同的查询维度</p><p>那么进入Python时代，我认为我们终于有机会跳脱Excel的框架，并以web框架为基础追求以下这些进步：</p><ul><li><strong>提升整体数据查询和分析的速度性能</strong></li><li><strong>提升多用户同时查询时的并发性能</strong></li><li><strong>提升交互操作的用户体验</strong></li><li><strong>在保留导出至Excel格式的同时提升结果可视化的整体用户体验，如提升表格和交互图表的美观性和便捷功能</strong></li></ul><p>在此基础上还有一些锦上添花的进阶需求，但我认为不是必须的：</p><ul><li>更好的用户权限系统</li><li>查询和展示功能的自定义扩展功能，如允许用户自助式编辑Dashboard视图，自选布局和图表类型</li><li>更丰富的导出功能，如导出到定制化的PDF Report</li></ul><p>本系列文章先主要考虑核心的功能需求。而为了尽可能高质量地实现这些功能，我在之前也考察了一些市面上的整合解决方案，开源项目中目前最接近我们需求的是Airbnb开源的superset，但是使用之后感觉还是灵活性有比较大欠缺。最终决定从零开始自己整合自制一个平台，我用到的技术实现如下：</p><ul><li>**后端Web框架：**Python 3.6 <strong>+</strong> Django 3.0.8</li><li>**数据库：**Mssql</li><li>**数据处理：**Pandas 1.0.5 + Numpy 1.19.0+mkl + 其他</li><li><strong>Javascript库:</strong> jQuery</li><li>**前端css：**Semantic UI 2.4</li><li>**前端交互图表：**Pyecharts 1.8.1</li><li>**前端数据表格：**jQuery Datatables</li><li>**部署：**Apache2.4 <strong>+</strong> mod_wsgi</li></ul><p>这套解决方案有很多地方都是可以斟酌替换的，如选择Django作为web框架对于一个自用小平台来说略显臃肿，可能Flask或FastAPI更灵活些，但是Django的文档最全生态健康，其实也挺适合我们这些二手程序员的。数据库使用Mssql是因为在老东家法国企业不允许用开源数据库，习惯沿用到了现在，Mysql或者Sqlite都可以替换。前端在当时没有使用React和Vue等更先进的框架，而是只使用了css，这是未来可以改进的方向。Semantic UI是我选择用来替代Bootstrap的。Pyecharts的替代方案则有highcharts等，但考虑echarts是国人出品可能对潜在的中国地图的展示需求更友好些。而Datatables的替代方案就更多了，数不胜数。可能唯独无可替代的就是Pandas+Numpy了，极大地提高了数据处理的效率，我真是太喜欢了。</p><p>那么，在我有勇气开始在下一篇进入正题之前，在本文末先看看实际效果吧。实际数据使用了医疗行业最著名的I司的外部销售数据，希望I司的人不会来找我麻烦。<br><img src="/images/python-djangosqlpand/v2-90b223605e9566aa9d00f502d886fc9e_1440w.webp"></p><p>第（二）篇，初步搭建Django环境，请移步：<br><img src="/images/python-djangosqlpand/v2-7511cd256edf21dd3673dad2deb5e3c7_1440w.webp"></p>]]>
    </content>
    <id>https://blog.malu.tech/python-django-sql-pandas-pyecharts-data-analysis-platform-1/</id>
    <link href="https://blog.malu.tech/python-django-sql-pandas-pyecharts-data-analysis-platform-1/"/>
    <published>2024-03-31T16:00:00.000Z</published>
    <summary>
      <![CDATA[Python Django+SQL+Pandas+Pyecharts自建在线数据分析平台系列文章第一篇：需求分析&技术实现]]>
    </summary>
    <title>Python Django+SQL+Pandas+Pyecharts自建在线数据分析平台（一）</title>
    <updated>2024-03-31T16:00:00.000Z</updated>
  </entry>
  <entry>
    <author>
      <name>Road</name>
    </author>
    <category term="Python" scheme="https://blog.malu.tech/categories/Python/"/>
    <category term="Django" scheme="https://blog.malu.tech/categories/Python/Django/"/>
    <category term="数据分析" scheme="https://blog.malu.tech/categories/Python/Django/%E6%95%B0%E6%8D%AE%E5%88%86%E6%9E%90/"/>
    <category term="Python" scheme="https://blog.malu.tech/tags/Python/"/>
    <category term="Django" scheme="https://blog.malu.tech/tags/Django/"/>
    <category term="SQL" scheme="https://blog.malu.tech/tags/SQL/"/>
    <category term="Pandas" scheme="https://blog.malu.tech/tags/Pandas/"/>
    <category term="Pyecharts" scheme="https://blog.malu.tech/tags/Pyecharts/"/>
    <category term="数据分析平台" scheme="https://blog.malu.tech/tags/%E6%95%B0%E6%8D%AE%E5%88%86%E6%9E%90%E5%B9%B3%E5%8F%B0/"/>
    <content>
      <![CDATA[<blockquote><p><strong>文章来源</strong>：<a href="https://zhuanlan.zhihu.com/p/142490087">Python Django+SQL+Pandas+Pyecharts自建在线数据分析平台（一）</a><br><strong>作者</strong>：<a href="https://www.zhihu.com/people/chen-cheng-76-40">ccpic</a><br><strong>感谢</strong>：感谢作者 ccpic 分享的优质内容，本网页主要用于学习知识的存档备份，欢迎点击原网页支持作者。</p></blockquote><p>（一）需求分析&amp;技术实现</p><p>（二）初步搭建Django环境</p><p>（三）页面布局&amp;Django模板</p><p>（四）SQL+Pandas初步处理数据</p><p>（五）前端表单交互</p><p>（六）Ajax异步传参与加载</p><p>（七）前端数据格式的处理</p><p>（八）DataTables接管前端表格</p><p>（九）Pyecharts实现交互图表</p><p>（十）静态图表的展示</p><p>（十一）“导出数据至Excel”功能</p><p>（十二）添加和配置缓存</p><p>（十三）用户登录系统</p><p>本文志在完成系列文章的最后一章。</p><p>部署到生产环境这回事以及背后的原理对我来说就真的是知识盲区了，只能做到知其然而不知其所以然。所以我们会很应用式地进行完这一章。</p><p>而且因为二手程序员的缘故，本文的部署环境也会以Windows为例。至于为什么选用Apache而不是别的Nginx什么的，纯粹是因为直接就用了，之前没有认真评估。在之后的学习和research中，能浅显地得出Apache更稳定而Nginx性能更好的说法，实际两者的技术差异还是蛮大的，但不在我的学习范围之内了。</p><p>总的来说部署Django有以下三大步骤：</p><p><strong>1、修改工程settings更符合生产环境</strong></p><p><strong>2、处理静态文件</strong></p><p><strong>3、配置Apache和mod_wsgi</strong></p><p>在我们工程settings.py文件里，原本有这样一串随机码：</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="comment"># SECURITY WARNING: keep the secret key used in production secret!</span></span><br><span class="line">SECRET_KEY = <span class="string">&#x27;qteh2xx_xz#z#keg0%*++%yo%)n2nn27!ogxk5#2z%4*k57^s)&#x27;</span></span><br></pre></td></tr></table></figure><p>这是用于CRSF token等的大随机值，生产环境中出于安全考虑不应该将其再置于settings.py中。</p><p>推荐的做法是在相同目录新建一个env.json文件，将SECRET_KEY写入此json文件中，包括其他一些隐私信息如服务器连接字符串也应置于此。而settings.py或其他需要调用处应改写为：</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">with</span> <span class="built_in">open</span> (<span class="string">&#x27;env.json&#x27;</span>, <span class="string">&#x27;r&#x27;</span>, encoding=<span class="string">&#x27;utf-8&#x27;</span>) <span class="keyword">as</span> env:</span><br><span class="line">    ENV_CONST = json.load(env)</span><br><span class="line"></span><br><span class="line">SECRET_KEY = ENV_CONST[<span class="string">&#x27;SECRET_KEY&#x27;</span>]</span><br></pre></td></tr></table></figure><p>接着，修改DEBUG变量：</p><figure class="highlight text"><table><tr><td class="code"><pre><span class="line">DEBUG = False</span><br></pre></td></tr></table></figure><p>并在ALLOWED_HOST中加入生产环境的域名或IP：</p><figure class="highlight text"><table><tr><td class="code"><pre><span class="line">ALLOWED_HOSTS = [&#x27;.xxx.com&#x27;, &#x27;xx.xx.xx.xx&#x27;, &#x27;127.0.0.1&#x27;, &#x27;localhost&#x27;]</span><br></pre></td></tr></table></figure><p>心大的可以这么写：</p><figure class="highlight text"><table><tr><td class="code"><pre><span class="line">ALLOWED_HOSTS = [&#x27;*&#x27;]</span><br></pre></td></tr></table></figure><p>接下来处理静态文件，我们需要在terminal运行一下下方的命令：</p><figure class="highlight text"><table><tr><td class="code"><pre><span class="line">python manage.py collectstatic</span><br></pre></td></tr></table></figure><p>但是可能会返回下面的错误：<br><img src="/images/python-djangosqlpand/v2-13c12e892a73495e2f9c8111e4d47689_1440w.webp"></p><p>根据末行的提示，很显然，我们需要在settings.py里再增加一个STATIC_ROOT：</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line">STATIC_ROOT = os.path.join(BASE_DIR + <span class="string">&quot;/static&quot;</span>)</span><br></pre></td></tr></table></figure><p>这时我们的settings里面可能同时有STATIC_URL，STATICFILES_DIRS和STATIC_ROOT，这让我们感到很困惑。他们的区别是：</p><ul><li>STATIC_ROOT是collectstatic命令把所有引用静态文件收集后目录的绝对位置</li><li>STATIC_URL则提供静态文件的基本 URL 位置，在本地或在CDN上。主要用在我们base模板中的{&#37; load static &#37;} tag</li><li>STATICFILES_DIRS不同的Django版本不一样，在很老的版本中起到的是STATIC_URL的作用，在更新的版本中似乎是当静态文件目录不是一个时，用来列出主目录外放置静态文件的其他目录</li></ul><p>增加了STATIC_ROOT后，别忘了正确运行一次collectstatic命令。</p><p>接下来开始安装mod_wsgi，windows环境下一定要在著名的<a href="https://www.lfd.uci.edu/~gohlke/pythonlibs/#mod\_wsgi%E4%B8%8B%E8%BD%BD%E5%AF%B9%E5%BA%94%E7%89%88%E6%9C%AC%EF%BC%8C%E5%86%8D%E7%94%A8pip%E6%89%8B%E5%8A%A8%E5%AE%89%E8%A3%85%E3%80%82">https://www.lfd.uci.edu/~gohlke/pythonlibs/#mod\_wsgi下载对应版本，再用pip手动安装。</a></p><p>安装成功后在terminal输入：</p><figure class="highlight text"><table><tr><td class="code"><pre><span class="line">mod_wsgi-express module-config</span><br></pre></td></tr></table></figure><p>复制下返回的字符串，这是将来要写入Apache配置文件中的。</p><p>最后安装并配置Apache，我用的是2.4版本。</p><p>下载后解压在一个文件夹内不用安装，类似一个portable的app。我们用任意编辑器编辑\Apache24\conf\httpd.conf这个文件。</p><p>首先修改Apache的配置部分：</p><figure class="highlight apacheconf"><table><tr><td class="code"><pre><span class="line"><span class="comment"># Apache文件夹的绝对位置</span></span><br><span class="line"><span class="attribute">Define</span> SRVROOT <span class="string">&quot;D:/web/Apache24&quot;</span> </span><br><span class="line"><span class="attribute">ServerRoot</span> <span class="string">&quot;$&#123;SRVROOT&#125;&quot;</span></span><br><span class="line"><span class="comment"># 填写监听端口</span></span><br><span class="line"><span class="attribute">Listen</span> <span class="number">8080</span></span><br><span class="line"><span class="comment"># ServerName填写域名，未备案的填写外网访问地址</span></span><br><span class="line"><span class="attribute">ServerName</span> xx.xx.xx.xx:<span class="number">8080</span></span><br></pre></td></tr></table></figure><p>接下来在文件末尾加入下方片段配置Django部分：</p><figure class="highlight apacheconf"><table><tr><td class="code"><pre><span class="line"><span class="comment">##--------------- Django项目部署配置 ---------------##</span></span><br><span class="line"><span class="comment"># 声明项目根目录变量，为了下方引用</span></span><br><span class="line"><span class="attribute">Define</span> DjangoRoot <span class="string">&quot;C:\Users\Administrator\PycharmProjects\datasite&quot;</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 添加&quot;mod_wsgi.so&quot;模块，这三行是之前mod_wsgi-express module-config命令返回的结果</span></span><br><span class="line"><span class="attribute">LoadFile</span> <span class="string">&quot;c:/users/administrator/appdata/local/programs/python/python38/python38.dll&quot;</span></span><br><span class="line"><span class="attribute">LoadModule</span> wsgi_module <span class="string">&quot;c:/users/administrator/appdata/local/programs/python/python38/lib/site-packages/mod_wsgi/server/mod_wsgi.cp38-win_amd64.pyd&quot;</span></span><br><span class="line"><span class="attribute">WSGIPythonHome</span> <span class="string">&quot;c:/users/administrator/appdata/local/programs/python/python38&quot;</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 指定项目的&quot;wsgi.py&quot;配置文件路径</span></span><br><span class="line"><span class="attribute">WSGIScriptAlias</span> / <span class="string">&quot;$&#123;DjangoRoot&#125;/datasite/wsgi.py&quot;</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 真正的指定Django项目根目录，并配置访问权限。</span></span><br><span class="line"><span class="attribute">WSGIPythonPath</span> <span class="string">&quot;$&#123;DjangoRoot&#125;&quot;</span></span><br><span class="line"><span class="section">&lt;Directory <span class="string">&quot;$&#123;DjangoRoot&#125;&quot;</span>&gt;</span></span><br><span class="line">    <span class="attribute">Require</span> <span class="literal">all</span> granted</span><br><span class="line"><span class="section">&lt;/Directory&gt;</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 项目静态文件配置 </span></span><br><span class="line"><span class="attribute">Alias</span> /static <span class="string">&quot;$&#123;DjangoRoot&#125;/static&quot;</span></span><br><span class="line"><span class="section">&lt;Directory <span class="string">&quot;$&#123;DjangoRoot&#125;/static&quot;</span>&gt;</span></span><br><span class="line">    <span class="attribute">AllowOverride</span> None</span><br><span class="line">    <span class="attribute">Options</span> None</span><br><span class="line">    <span class="attribute">Require</span> <span class="literal">all</span> granted</span><br><span class="line"><span class="section">&lt;/Directory&gt;</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 项目media文件配置, 用户上传图片等媒体文件</span></span><br><span class="line"><span class="attribute">Alias</span> /media <span class="string">&quot;$&#123;DjangoRoot&#125;/media&quot;</span></span><br><span class="line"><span class="section">&lt;Directory <span class="string">&quot;$&#123;DjangoRoot&#125;/media&quot;</span>&gt;</span></span><br><span class="line">    <span class="attribute">AllowOverride</span> None  </span><br><span class="line">    <span class="attribute">Options</span> None  </span><br><span class="line">    <span class="attribute">Require</span> <span class="literal">all</span> granted  </span><br><span class="line"><span class="section">&lt;/Directory&gt;</span></span><br></pre></td></tr></table></figure><p>这里有个Apache24配置Django含Pandas包时的bug，如果上方配置完毕后测试访问出现无响应，请在配置里再加上下面这句：</p><figure class="highlight text"><table><tr><td class="code"><pre><span class="line">WSGIApplicationGroup %&#123;GLOBAL&#125;</span><br></pre></td></tr></table></figure><p>配置好文件后，在\Apache24\bin下，可以安装Apache服务：</p><figure class="highlight text"><table><tr><td class="code"><pre><span class="line">httpd -k install -n Apache</span><br></pre></td></tr></table></figure><p>或者临时启动一次Apache：</p><figure class="highlight text"><table><tr><td class="code"><pre><span class="line">httpd</span><br></pre></td></tr></table></figure><p>我们的部署都可以宣告完成了~</p>]]>
    </content>
    <id>https://blog.malu.tech/python-django-sql-pandas-pyecharts-data-analysis-platform-14/</id>
    <link href="https://blog.malu.tech/python-django-sql-pandas-pyecharts-data-analysis-platform-14/"/>
    <published>2024-03-31T16:00:00.000Z</published>
    <summary>Python Django+SQL+Pandas+Pyecharts自建在线数据分析平台系列文章第十四篇：部署Django至生产环境</summary>
    <title>Python Django+SQL+Pandas+Pyecharts自建在线数据分析平台（十四）</title>
    <updated>2024-03-31T16:00:00.000Z</updated>
  </entry>
  <entry>
    <author>
      <name>Road</name>
    </author>
    <category term="Python" scheme="https://blog.malu.tech/categories/Python/"/>
    <category term="Django" scheme="https://blog.malu.tech/categories/Python/Django/"/>
    <category term="数据分析" scheme="https://blog.malu.tech/categories/Python/Django/%E6%95%B0%E6%8D%AE%E5%88%86%E6%9E%90/"/>
    <category term="Python" scheme="https://blog.malu.tech/tags/Python/"/>
    <category term="Django" scheme="https://blog.malu.tech/tags/Django/"/>
    <category term="SQL" scheme="https://blog.malu.tech/tags/SQL/"/>
    <category term="Pandas" scheme="https://blog.malu.tech/tags/Pandas/"/>
    <category term="Pyecharts" scheme="https://blog.malu.tech/tags/Pyecharts/"/>
    <category term="数据分析平台" scheme="https://blog.malu.tech/tags/%E6%95%B0%E6%8D%AE%E5%88%86%E6%9E%90%E5%B9%B3%E5%8F%B0/"/>
    <content>
      <![CDATA[<blockquote><p><strong>文章来源</strong>：<a href="https://zhuanlan.zhihu.com/p/142490087">Python Django+SQL+Pandas+Pyecharts自建在线数据分析平台（一）</a><br><strong>作者</strong>：<a href="https://www.zhihu.com/people/chen-cheng-76-40">ccpic</a><br><strong>感谢</strong>：感谢作者 ccpic 分享的优质内容，本网页主要用于学习知识的存档备份，欢迎点击原网页支持作者。</p></blockquote><p>（一）需求分析&amp;技术实现</p><p>（二）初步搭建Django环境</p><p>（三）页面布局&amp;Django模板</p><p>（四）SQL+Pandas初步处理数据</p><p>（五）前端表单交互</p><p>（六）Ajax异步传参与加载</p><p>（七）前端数据格式的处理</p><p>（八）DataTables接管前端表格</p><p>（九）Pyecharts实现交互图表</p><p>（十）静态图表的展示</p><p>（十一）“导出数据至Excel”功能</p><p>（十二）添加和配置缓存</p><h2 id="（十三）用户登录系统"><a href="#（十三）用户登录系统" class="headerlink" title="（十三）用户登录系统"></a><strong>（十三）用户登录系统</strong></h2><p>（十四）部署Django至生产环境</p><p>本章我们的目标是完成一个用户登录系统，使数据不至于裸奔。我们的需求是比较简单的，因为是自建自用的数据平台，用户权限可以在后台分配，也不涉及注册模块和密码找回等功能，就是简单的登录和登出。</p><p>Django有比较完善的原生用户和授权系统，包括内置的User和Group模型。一般情况下，我们不需要重写和扩展这些模型，直接应用即可。</p><p>首先，确保在工程的settings.py文件中有下方与用户权限相关的installed app和中间件：</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line">INSTALLED_APPS = [</span><br><span class="line">    ...</span><br><span class="line">![](images/python-djangosqlpand/v2-fd21b983e4e862ec45479c1937087269_1440w.webp)</span><br><span class="line">    <span class="string">&#x27;django.contrib.auth&#x27;</span>,</span><br><span class="line">    <span class="string">&#x27;django.contrib.contenttypes&#x27;</span>,</span><br><span class="line"></span><br><span class="line">MIDDLEWARE = [</span><br><span class="line">    ...</span><br><span class="line">    <span class="string">&#x27;django.contrib.sessions.middleware.SessionMiddleware&#x27;</span>,</span><br><span class="line">    <span class="string">&#x27;django.contrib.auth.middleware.AuthenticationMiddleware&#x27;</span>,</span><br><span class="line">    ]</span><br></pre></td></tr></table></figure><p>Django原生系统的用户数据和sesssion数据都需要数据库存储，我们之前一直没有使用Django的数据库接口，这次要用了。我们可以使用django默认的sqlite3数据库，它是个单文件轻型数据库，与业务数据分离。settings.py里设置如下：</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line">DATABASES = &#123;</span><br><span class="line">    <span class="string">&#x27;default&#x27;</span>: &#123;</span><br><span class="line">        <span class="string">&#x27;ENGINE&#x27;</span>: <span class="string">&#x27;django.db.backends.sqlite3&#x27;</span>,</span><br><span class="line">        <span class="string">&#x27;NAME&#x27;</span>: os.path.join(BASE_DIR, <span class="string">&#x27;db.sqlite3&#x27;</span>),</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>因为是第一次使用Django数据库接口，我们需要执行migrate命令一次。如果在一般的Django项目中，migrate命令大概我们很早就很熟识了：</p><figure class="highlight text"><table><tr><td class="code"><pre><span class="line">python manage.py migrate</span><br></pre></td></tr></table></figure><p>再顺便把超级管理员账号创建了</p><figure class="highlight text"><table><tr><td class="code"><pre><span class="line">python manage.py createsuperuser</span><br></pre></td></tr></table></figure><p>其实我们进行到这个步骤时会有一种强烈感觉，我们依然在以Django MTV的的框架思路在搭建这个用户系统。上面完成的工作是M的部分，并且我们已经有了一条符合内置User model的超级管理员用户信息的数据。下面我们需要继续完成T和V的部分。</p><p>在template下新建registration文件夹，里面再新建login.html模板。<br><img src="/images/python-djangosqlpand/v2-f52d7a7f71ac1805256f6cd303729b1a_1440w.webp"></p><p>login模板还是继承自base.html，并且内容部分以**{&#37; block body &#37;}{&#37; endblock &#37;}**开头结尾。在第三章我们解释过这种模板继承的机制，并且这个login页面就和我们在第三章举的index页面的例子一模一样。</p><p>而login.html内容为：</p><figure class="highlight html"><table><tr><td class="code"><pre><span class="line"><span class="comment">&lt;!-- extends表明此页面继承自 base.html 文件 --&gt;</span></span><br><span class="line">&#123;<span class="symbol">&amp;#37;</span> extends &quot;chpa_data/base.html&quot; <span class="symbol">&amp;#37;</span>&#125;</span><br><span class="line"></span><br><span class="line">&#123;<span class="symbol">&amp;#37;</span> block body <span class="symbol">&amp;#37;</span>&#125;</span><br><span class="line"><span class="tag">&lt;<span class="name">div</span> <span class="attr">id</span>=<span class="string">&quot;pusher&quot;</span> <span class="attr">class</span>=<span class="string">&quot;pusher&quot;</span> <span class="attr">style</span>=<span class="string">&quot;padding-top:100px&quot;</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;ui middle aligned center aligned grid&quot;</span>&gt;</span></span><br><span class="line">        <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;column&quot;</span>&gt;</span></span><br><span class="line">            <span class="tag">&lt;<span class="name">h2</span> <span class="attr">class</span>=<span class="string">&quot;ui image header&quot;</span>&gt;</span></span><br><span class="line">                <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;content&quot;</span>&gt;</span></span><br><span class="line">                    用户登录</span><br><span class="line">                <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">            <span class="tag">&lt;/<span class="name">h2</span>&gt;</span></span><br><span class="line"></span><br><span class="line">            <span class="tag">&lt;<span class="name">form</span> <span class="attr">method</span>=<span class="string">&quot;post&quot;</span> <span class="attr">action</span>=<span class="string">&quot;&#123;<span class="symbol">&amp;#37;</span> url &#x27;login&#x27; <span class="symbol">&amp;#37;</span>&#125;&quot;</span> <span class="attr">class</span>=<span class="string">&quot;ui large form&quot;</span>&gt;</span></span><br><span class="line">                &#123;<span class="symbol">&amp;#37;</span> csrf_token <span class="symbol">&amp;#37;</span>&#125;</span><br><span class="line">                &#123;<span class="symbol">&amp;#37;</span> if next <span class="symbol">&amp;#37;</span>&#125;</span><br><span class="line">                    &#123;<span class="symbol">&amp;#37;</span> if user.is_authenticated <span class="symbol">&amp;#37;</span>&#125;</span><br><span class="line">                        <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;ui info message&quot;</span>&gt;</span>您的账户没有权限浏览当前页面。 请尝试登录有权限的账号<span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">                    &#123;<span class="symbol">&amp;#37;</span> else <span class="symbol">&amp;#37;</span>&#125;</span><br><span class="line">                        <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;ui info message&quot;</span>&gt;</span>未登录用户没有权限浏览当前页面，请登录<span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">                    &#123;<span class="symbol">&amp;#37;</span> endif <span class="symbol">&amp;#37;</span>&#125;</span><br><span class="line">                &#123;<span class="symbol">&amp;#37;</span> endif <span class="symbol">&amp;#37;</span>&#125;</span><br><span class="line">                <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;ui stacked secondary  segment&quot;</span>&gt;</span></span><br><span class="line">                    <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;field&quot;</span>&gt;</span></span><br><span class="line">                        <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;ui left icon input&quot;</span>&gt;</span></span><br><span class="line">                            <span class="tag">&lt;<span class="name">i</span> <span class="attr">class</span>=<span class="string">&quot;user icon&quot;</span>&gt;</span><span class="tag">&lt;/<span class="name">i</span>&gt;</span></span><br><span class="line">                            &#123;<span class="symbol">&amp;#123;</span> form.username <span class="symbol">&amp;#125;</span>&#125;</span><br><span class="line">                        <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">                    <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">                    <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;field&quot;</span>&gt;</span></span><br><span class="line">                        <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;ui left icon input&quot;</span>&gt;</span></span><br><span class="line">                            <span class="tag">&lt;<span class="name">i</span> <span class="attr">class</span>=<span class="string">&quot;lock icon&quot;</span>&gt;</span><span class="tag">&lt;/<span class="name">i</span>&gt;</span></span><br><span class="line">                            &#123;<span class="symbol">&amp;#123;</span> form.password <span class="symbol">&amp;#125;</span>&#125;</span><br><span class="line">                        <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">                    <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">                    <span class="tag">&lt;<span class="name">input</span> <span class="attr">class</span>=<span class="string">&quot;ui fluid large blue submit button&quot;</span> <span class="attr">type</span>=<span class="string">&quot;submit&quot;</span> <span class="attr">value</span>=<span class="string">&quot;登录&quot;</span>&gt;</span></span><br><span class="line">                    <span class="tag">&lt;<span class="name">input</span> <span class="attr">type</span>=<span class="string">&quot;hidden&quot;</span> <span class="attr">name</span>=<span class="string">&quot;next&quot;</span> <span class="attr">value</span>=<span class="string">&quot;&#123;<span class="symbol">&amp;#123;</span> next <span class="symbol">&amp;#125;</span>&#125;&quot;</span>&gt;</span></span><br><span class="line">                <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">                &#123;<span class="symbol">&amp;#37;</span> if form.errors <span class="symbol">&amp;#37;</span>&#125;</span><br><span class="line">                <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;ui info message&quot;</span>&gt;</span>用户名或密码错误，请再次尝试。<span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">                &#123;<span class="symbol">&amp;#37;</span> endif <span class="symbol">&amp;#37;</span>&#125;</span><br><span class="line"></span><br><span class="line">            <span class="tag">&lt;/<span class="name">form</span>&gt;</span></span><br><span class="line"></span><br><span class="line">            <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;ui message&quot;</span>&gt;</span></span><br><span class="line">                如登录困难，请联系管理员</span><br><span class="line">            <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">        <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line"><span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">...</span><br><span class="line">&#123;<span class="symbol">&amp;#37;</span> endblock body <span class="symbol">&amp;#37;</span>&#125;</span><br></pre></td></tr></table></figure><p>此时登录界面已经有了，但是布局有点诡异。<br><img src="/images/python-djangosqlpand/v2-67245a4258682ce322c97726cd983251_1440w.webp"></p><p>我们需要写一些css调整下：</p><figure class="highlight css"><table><tr><td class="code"><pre><span class="line">&#123;&amp;<span class="selector-id">#37</span>; block <span class="selector-tag">body</span> &amp;<span class="selector-id">#37</span>;&#125;</span><br><span class="line">...</span><br><span class="line">&lt;style&gt;</span><br><span class="line">    <span class="selector-tag">body</span> &#123;</span><br><span class="line">        <span class="attribute">background-color</span>: <span class="number">#DADADA</span>;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="selector-tag">body</span> &gt; <span class="selector-class">.grid</span> &#123;</span><br><span class="line">        <span class="attribute">height</span>: <span class="number">100%</span>;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="selector-class">.column</span> &#123;</span><br><span class="line">        <span class="attribute">max-width</span>: <span class="number">450px</span>;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="selector-class">.ui</span><span class="selector-class">.footer</span><span class="selector-class">.segment</span> &#123;</span><br><span class="line">        <span class="attribute">margin</span>: <span class="number">5em</span> <span class="number">0em</span> <span class="number">0em</span>;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">&lt;/style&gt;</span><br><span class="line">&#123;&amp;<span class="selector-id">#37</span>; endblock <span class="selector-tag">body</span> &amp;<span class="selector-id">#37</span>;&#125;</span><br></pre></td></tr></table></figure><p>Login.html的代码很好读懂，只有一个{&#123; next &#125;}有些奇怪，但我们放一放，到后面相关部分再做解释。</p><p>下面配置URL，我们需要更改的是工程的urls.py，而不是chpa app的urls.py。添加登录权限相关url后它应该长这样：</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line">urlpatterns = [</span><br><span class="line"></span><br><span class="line">    path(<span class="string">&#x27;chpa/&#x27;</span>, include(<span class="string">&#x27;chpa_data.urls&#x27;</span>)),</span><br><span class="line">    path(<span class="string">&#x27;admin/&#x27;</span>, admin.site.urls),</span><br><span class="line">    path(<span class="string">&#x27;accounts/&#x27;</span>, include(<span class="string">&#x27;django.contrib.auth.urls&#x27;</span>)),</span><br><span class="line">]</span><br></pre></td></tr></table></figure><p>同时我们要在工程的settings.py里设置2个参数，明确登入和登出成功后分别的默认landing page：</p><figure class="highlight text"><table><tr><td class="code"><pre><span class="line">LOGIN_REDIRECT_URL = &#x27;/chpa/index&#x27;</span><br><span class="line">LOGOUT_REDIRECT_URL = &#x27;/accounts/login&#x27;</span><br></pre></td></tr></table></figure><p>此时我们如果访问<a href="http://127.0.0.1:8088/accounts/login/%EF%BC%8C%E5%B7%B2%E7%BB%8F%E8%83%BD%E4%BD%BF%E7%94%A8%E8%BF%99%E4%B8%AA%E7%99%BB%E5%BD%95%E7%95%8C%E9%9D%A2%E4%BA%86%EF%BC%8C%E6%88%91%E4%BB%AC%E7%94%9A%E8%87%B3%E5%8F%AF%E4%BB%A5%E8%BE%93%E5%85%A5%E4%B9%8B%E5%89%8D%E8%AE%BE%E7%BD%AE%E7%9A%84%E8%B6%85%E7%BA%A7%E7%AE%A1%E7%90%86%E5%91%98%E8%B4%A6%E6%88%B7%EF%BC%8C%E7%99%BB%E5%BD%95%E6%88%90%E5%8A%9F%E5%90%8E%E9%87%8D%E5%AE%9A%E5%90%91%E5%88%B0%E4%B8%8A%E6%96%B9%E7%9A%84/chpa/index%E9%A1%B5%E9%9D%A2%E3%80%82">http://127.0.0.1:8088/accounts/login/，已经能使用这个登录界面了，我们甚至可以输入之前设置的超级管理员账户，登录成功后重定向到上方的/chpa/index页面。</a></p><p>但是，现在直接输入其他url能绕过此登录验证，我们还需要给所有与URL绑定的视图方法（本例中也就是views.py里的index, query, search, export4个）都加上@login_required装饰器， 如果涉及到多个装饰器的场合，@login_required装饰器放在最上面第一个加载：</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">from</span> django.contrib.auth.decorators <span class="keyword">import</span> login_required</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="meta">@login_required</span></span><br><span class="line"><span class="keyword">def</span> <span class="title function_">index</span>(<span class="params">request</span>):</span><br><span class="line">    ...</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="meta">@login_required</span></span><br><span class="line"><span class="meta">@cache_page(<span class="params"><span class="number">60</span> * <span class="number">60</span> * <span class="number">24</span> * <span class="number">30</span></span>)</span></span><br><span class="line"><span class="keyword">def</span> <span class="title function_">query</span>(<span class="params">request</span>):</span><br><span class="line">    ...</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="meta">@login_required</span></span><br><span class="line"><span class="meta">@cache_page(<span class="params"><span class="number">60</span> * <span class="number">60</span> * <span class="number">24</span> * <span class="number">30</span></span>)</span></span><br><span class="line"><span class="keyword">def</span> <span class="title function_">search</span>(<span class="params">request, column, kw</span>):</span><br><span class="line">    ...</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="meta">@login_required</span></span><br><span class="line"><span class="meta">@cache_page(<span class="params"><span class="number">60</span> * <span class="number">60</span> * <span class="number">24</span> * <span class="number">30</span></span>)</span></span><br><span class="line"><span class="keyword">def</span> <span class="title function_">export</span>(<span class="params">request, <span class="built_in">type</span></span>):</span><br><span class="line">    ...</span><br></pre></td></tr></table></figure><p>此时可以回过头来解释login.html里{&#123; next &#125;}的问题。</p><p>以query方法为例，@login_required装饰器会在query方法之前运行，即当任何URL成功定向到query方法后，@login_required将首先检查该用户是否已登录。如果他们已登录，则query将运行并返回结果。</p><p>而如果未登录，它将阻止query方法运行，而重定向至&#x2F;login?next&#x3D;%2Fquery。加载页面后此时的next后的参数为一个GET请求的参数被捕捉。所以这时login.html可以用django tag语法调用{&#123; next &#125;}</p><p>login.html再使用<input type="hidden" name="next" value="{&#123; next &#125;}">语句，把next作为一个隐藏元素在登录时发送，用户登录成功后此next相当于覆盖了默认的LOGIN_REDIRECT_URL ，把页面重定向回query方法。</p><p>简单地说，这个{&#123; next &#125;}可以帮助我们在登录后重定向至登录前访问的网页。</p><p>最后，我们做一些收尾工作。因为我们需要Django自带的admin界面至少完成用户管理的工作，我们在header做一些admin的超链接以及登出的按钮等工作。此时的header.html为：</p><figure class="highlight html"><table><tr><td class="code"><pre><span class="line"><span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;ui large top fixed inverted menu&quot;</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;ui container&quot;</span>&gt;</span></span><br><span class="line">        <span class="tag">&lt;<span class="name">a</span> <span class="attr">href</span>=<span class="string">&quot;&#123;<span class="symbol">&amp;#37;</span> url &#x27;chpa:index&#x27; <span class="symbol">&amp;#37;</span>&#125;&quot;</span> <span class="attr">class</span>=<span class="string">&quot;item&quot;</span>&gt;</span>首页<span class="tag">&lt;/<span class="name">a</span>&gt;</span></span><br><span class="line">        &#123;<span class="symbol">&amp;#37;</span>  if request.user.is_staff  <span class="symbol">&amp;#37;</span>&#125;</span><br><span class="line">            <span class="tag">&lt;<span class="name">a</span> <span class="attr">href</span>=<span class="string">&quot;&#123;<span class="symbol">&amp;#37;</span> url &#x27;admin:index&#x27; <span class="symbol">&amp;#37;</span>&#125;&quot;</span> <span class="attr">class</span>=<span class="string">&quot;item&quot;</span>&gt;</span>后台管理<span class="tag">&lt;/<span class="name">a</span>&gt;</span></span><br><span class="line">        &#123;<span class="symbol">&amp;#37;</span> endif <span class="symbol">&amp;#37;</span>&#125;</span><br><span class="line">    <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;right menu&quot;</span>&gt;</span></span><br><span class="line">        &#123;<span class="symbol">&amp;#37;</span>  if not request.user.username  <span class="symbol">&amp;#37;</span>&#125;</span><br><span class="line">            <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;ui item&quot;</span>&gt;</span></span><br><span class="line">                未登录</span><br><span class="line">            <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">        &#123;<span class="symbol">&amp;#37;</span> else <span class="symbol">&amp;#37;</span>&#125;</span><br><span class="line">            <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;ui item&quot;</span>&gt;</span></span><br><span class="line">                您好，&#123;<span class="symbol">&amp;#123;</span> request.user.username <span class="symbol">&amp;#125;</span>&#125;</span><br><span class="line">            <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">            <span class="tag">&lt;<span class="name">a</span> <span class="attr">class</span>=<span class="string">&quot;ui item&quot;</span> <span class="attr">href</span>=<span class="string">&quot;&#123;<span class="symbol">&amp;#37;</span> url &#x27;logout&#x27; <span class="symbol">&amp;#37;</span>&#125;&quot;</span>&gt;</span></span><br><span class="line">                切换用户</span><br><span class="line">            <span class="tag">&lt;/<span class="name">a</span>&gt;</span></span><br><span class="line">        &#123;<span class="symbol">&amp;#37;</span> endif <span class="symbol">&amp;#37;</span>&#125;</span><br><span class="line">  <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line"><span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br></pre></td></tr></table></figure><p>完成~<br><img src="/images/python-djangosqlpand/v2-62664ebab68a0f2ec529ba2615e6720c_1440w.webp"></p><p>增加了后台管理链接和右上角（显示当前用户名和登出按钮）</p>]]>
    </content>
    <id>https://blog.malu.tech/python-django-sql-pandas-pyecharts-data-analysis-platform-13/</id>
    <link href="https://blog.malu.tech/python-django-sql-pandas-pyecharts-data-analysis-platform-13/"/>
    <published>2024-03-24T16:00:00.000Z</published>
    <summary>Python Django+SQL+Pandas+Pyecharts自建在线数据分析平台系列文章第十三篇：用户登录系统</summary>
    <title>Python Django+SQL+Pandas+Pyecharts自建在线数据分析平台（十三）</title>
    <updated>2024-03-24T16:00:00.000Z</updated>
  </entry>
  <entry>
    <author>
      <name>Road</name>
    </author>
    <category term="Python" scheme="https://blog.malu.tech/categories/Python/"/>
    <category term="Django" scheme="https://blog.malu.tech/categories/Python/Django/"/>
    <category term="数据分析" scheme="https://blog.malu.tech/categories/Python/Django/%E6%95%B0%E6%8D%AE%E5%88%86%E6%9E%90/"/>
    <category term="Python" scheme="https://blog.malu.tech/tags/Python/"/>
    <category term="Django" scheme="https://blog.malu.tech/tags/Django/"/>
    <category term="SQL" scheme="https://blog.malu.tech/tags/SQL/"/>
    <category term="Pandas" scheme="https://blog.malu.tech/tags/Pandas/"/>
    <category term="Pyecharts" scheme="https://blog.malu.tech/tags/Pyecharts/"/>
    <category term="数据分析平台" scheme="https://blog.malu.tech/tags/%E6%95%B0%E6%8D%AE%E5%88%86%E6%9E%90%E5%B9%B3%E5%8F%B0/"/>
    <content>
      <![CDATA[<p><img src="/images/python-djangosqlpand/v2-575478fd2109b9cc61c6f4d74075878c_1440w.webp"></p><blockquote><p><strong>文章来源</strong>：<a href="https://zhuanlan.zhihu.com/p/142490087">Python Django+SQL+Pandas+Pyecharts自建在线数据分析平台（一）</a><br><strong>作者</strong>：<a href="https://www.zhihu.com/people/chen-cheng-76-40">ccpic</a><br><strong>感谢</strong>：感谢作者 ccpic 分享的优质内容，本网页主要用于学习知识的存档备份，欢迎点击原网页支持作者。</p></blockquote><p>（一）需求分析&amp;技术实现</p><p>（二）初步搭建Django环境</p><p>（三）页面布局&amp;Django模板</p><p>（四）SQL+Pandas初步处理数据</p><p>（五）前端表单交互</p><p>（六）Ajax异步传参与加载</p><p>（七）前端数据格式的处理</p><p>（八）DataTables接管前端表格</p><p>（九）Pyecharts实现交互图表</p><p>（十）静态图表的展示</p><p>（十一）“导出数据至Excel”功能</p><h2 id="（十二）添加和配置缓存"><a href="#（十二）添加和配置缓存" class="headerlink" title="（十二）添加和配置缓存"></a><strong>（十二）添加和配置缓存</strong></h2><p>（十三）用户登录系统</p><p>（十四）部署Django至生产环境</p><p>缓存机制相当容易理解，无非就是系统在第一次请求时把某段时间内不会发生改变的返回内容存储在更快的介质当中，一段时间内相同的后续请求动作就不需要再重复完整的后端工作，而可以直接从缓存中读取结果。</p><p>落实到Django层面上，大部分场景下缓存机制发生在最前端，在模板层和视图层之前，也就是Django的缓存能同时减少重复的数据库请求、视图层的计算以及渲染的过程。</p><p>无疑，缓存机制能大大加强系统的性能并同时提升用户体验，但并不是任何场景都适用。如果后台频繁地更新数据，缓存会变得没有意义，且还可能返回某种意义上“错误的“（过时的）response。</p><p>**缓存最适合的场景是前端请求频率高+后台更新频率低。**在类似我们这个项目的数据查询项目中，后台的数据更新频率完全取决于行业的不同和数据业务的方向。对于我用来测试的这版等级医院药物销售数据，是非实时更新的月度收集数据，缓存的价值是很大的。</p><p>Django支持多种类型的缓存，包括Memcached, Redis，数据库，文件系统，本地内存等等……官方文档是最推荐Memcached的，但在实际操作中我发现Memcached对windows环境支持不佳，所以本章以文件缓存为例。</p><p>先在工程的setting.py文件（注意不是app的，是整个工程的setting）中加入缓存的设置语句：</p><figure class="highlight text"><table><tr><td class="code"><pre><span class="line">CACHES = &#123;</span><br><span class="line">    &#x27;default&#x27;: &#123;</span><br><span class="line">        &#x27;BACKEND&#x27;: &#x27;django.core.cache.backends.filebased.FileBasedCache&#x27;,</span><br><span class="line">        &#x27;LOCATION&#x27;: &#x27;D:/foo/bar&#x27;, # 缓存文件存放文件夹，需要有读写权限</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>具体操作时Django又分为以下5种缓存对象应用到不同内容：</p><ul><li><strong>整站缓存</strong></li><li><strong>视图缓存</strong></li><li><strong>URL缓存</strong></li><li><strong>模板碎片缓存</strong></li><li><strong>low-level API缓存</strong></li></ul><p>其中视图缓存和URL缓存应用的场景较多，这两者在大部分时候是等价没区别的。</p><p>视图缓存和URL缓存都要先引用缓存装饰器：</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">from</span> django.views.decorators.cache <span class="keyword">import</span> cache_page</span><br></pre></td></tr></table></figure><p>缓存装饰器可以放在views.py核心方法的前面，而最重要的是需要声明duration，以秒数为单位。这个值应该<strong>充分考虑后台数据的更新频率</strong>，并且可以使用公式的方式更加明确这个duration到底是什么：</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="meta">@cache_page(<span class="params"><span class="number">60</span> * <span class="number">60</span> * <span class="number">24</span> * <span class="number">30</span></span>) </span><span class="comment">#  缓存30天</span></span><br><span class="line"><span class="keyword">def</span> <span class="title function_">query</span>(<span class="params">request</span>):</span><br><span class="line">    ...</span><br><span class="line">    <span class="keyword">return</span> HttpResponse(json.dumps(context, ensure_ascii=<span class="literal">False</span>), content_type=<span class="string">&quot;application/json charset=utf-8&quot;</span>) <span class="comment"># 返回结果必须是json格式</span></span><br></pre></td></tr></table></figure><p>视图缓存和URL缓存代码非常简单，需要添加的语句不多。但需要你的返回内容相对每个参数是完全静态的。如果整站缓存或模板层缓存操作需要稍微复杂点，一个要添加中间件，一个要标识缓存DOM。Low-level API缓存操作最为复杂，需要object-by-object地操作。</p><p>这时再进行查询，可以观察到缓存文件夹里实时新生成的缓存文件：<br><img src="/images/python-djangosqlpand/v2-ca7fc05afc009425ae4bdd631086dbf7_1440w.webp"></p><p>实际操作中，缓存过的页面再查询的返回速度变成了毫秒级别，效果很明显。<br><img src="/images/python-djangosqlpand/v2-541a832cef6c6e58de6b7ef671fc23ab_1440w.webp"></p><p><img src="/images/python-djangosqlpand/v2-541a832cef6c6e58de6b7ef671fc23ab_1440w.webp"></p><p>Django的缓存在应用层面很简单，可以折腾的花样不多。唯一还需要注意的一点是——缓存机制可能对开发环境的测试有所不便，建议开发时把缓存的backend改成下方的dummy。它仅仅实现了缓存的接口而不做任何实际的事情，可以在不修改缓存代码的情况下正常开发测试。</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line">CACHES = &#123;</span><br><span class="line">    <span class="string">&#x27;default&#x27;</span>: &#123;</span><br><span class="line">        <span class="string">&#x27;BACKEND&#x27;</span>: <span class="string">&#x27;django.core.cache.backends.dummy.DummyCache&#x27;</span>,</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>如在测试期间遇到经常需要手动删除缓存的情况，可以在app\management\command下建一个clearcache.py（当然别忘了空白的_init_.py）<br><img src="/images/python-djangosqlpand/v2-f608f71fe2accab4c2f6ae2e46fa35ac_1440w.webp"></p><p>clearcache.py内容为：</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">from</span> django.core.management.base <span class="keyword">import</span> BaseCommand</span><br><span class="line"><span class="keyword">from</span> django.core.cache <span class="keyword">import</span> cache</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="keyword">class</span> <span class="title class_">Command</span>(<span class="title class_ inherited__">BaseCommand</span>):</span><br><span class="line">    <span class="keyword">def</span> <span class="title function_">handle</span>(<span class="params">self, *args, **kwargs</span>):</span><br><span class="line">        cache.clear()</span><br><span class="line">        <span class="variable language_">self</span>.stdout.write(<span class="string">&#x27;缓存已清除\n&#x27;</span>)</span><br></pre></td></tr></table></figure><p>之后即可在terminal手动输入命令清除缓存</p><figure class="highlight text"><table><tr><td class="code"><pre><span class="line">python manage.py clearcache</span><br></pre></td></tr></table></figure>]]>
    </content>
    <id>https://blog.malu.tech/python-django-sql-pandas-pyecharts-data-analysis-platform-12/</id>
    <link href="https://blog.malu.tech/python-django-sql-pandas-pyecharts-data-analysis-platform-12/"/>
    <published>2024-03-17T16:00:00.000Z</published>
    <summary>Python Django+SQL+Pandas+Pyecharts自建在线数据分析平台系列文章第十二篇：添加和配置缓存</summary>
    <title>Python Django+SQL+Pandas+Pyecharts自建在线数据分析平台（十二）</title>
    <updated>2024-03-17T16:00:00.000Z</updated>
  </entry>
  <entry>
    <author>
      <name>Road</name>
    </author>
    <category term="Python" scheme="https://blog.malu.tech/categories/Python/"/>
    <category term="Django" scheme="https://blog.malu.tech/categories/Python/Django/"/>
    <category term="数据分析" scheme="https://blog.malu.tech/categories/Python/Django/%E6%95%B0%E6%8D%AE%E5%88%86%E6%9E%90/"/>
    <category term="Python" scheme="https://blog.malu.tech/tags/Python/"/>
    <category term="Django" scheme="https://blog.malu.tech/tags/Django/"/>
    <category term="SQL" scheme="https://blog.malu.tech/tags/SQL/"/>
    <category term="Pandas" scheme="https://blog.malu.tech/tags/Pandas/"/>
    <category term="Pyecharts" scheme="https://blog.malu.tech/tags/Pyecharts/"/>
    <category term="数据分析平台" scheme="https://blog.malu.tech/tags/%E6%95%B0%E6%8D%AE%E5%88%86%E6%9E%90%E5%B9%B3%E5%8F%B0/"/>
    <content>
      <![CDATA[<blockquote><p><strong>文章来源</strong>：<a href="https://zhuanlan.zhihu.com/p/142490087">Python Django+SQL+Pandas+Pyecharts自建在线数据分析平台（一）</a><br><strong>作者</strong>：<a href="https://www.zhihu.com/people/chen-cheng-76-40">ccpic</a><br><strong>感谢</strong>：感谢作者 ccpic 分享的优质内容，本网页主要用于学习知识的存档备份，欢迎点击原网页支持作者。</p></blockquote><p>（一）需求分析&amp;技术实现</p><p>（二）初步搭建Django环境</p><p>（三）页面布局&amp;Django模板</p><p>（四）SQL+Pandas初步处理数据</p><p>（五）前端表单交互</p><p>（六）Ajax异步传参与加载</p><p>（七）前端数据格式的处理</p><p>（八）DataTables接管前端表格</p><p>（九）Pyecharts实现交互图表</p><p>（十）静态图表的展示</p><h2 id="（十一）“导出数据至Excel”功能"><a href="#（十一）“导出数据至Excel”功能" class="headerlink" title="（十一）“导出数据至Excel”功能"></a>（十一）“导出数据至Excel”功能</h2><p>（十二）添加和配置缓存</p><p>（十三）用户登录系统</p><p>（十四）部署Django至生产环境</p><p>本章实现最后一个必须的功能“导出数据至Excel”，因为目标很明确，实现是比较简单直接的，但也有一些需要注意的地方。</p><p>首先，我们需要明确导出数据的response这次不再适合用AJAX回调函数调用了，我们的整个导出功能需要和查询功能query方法并行，也就是需要完全新建一个方法以及相应URL。</p><p>为了复用，我们先把之前views.py文件里query方法的前一部分改写成get_df方法：</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">def</span> <span class="title function_">get_df</span>(<span class="params">form_dict, is_pivoted=<span class="literal">True</span></span>):</span><br><span class="line">    sql = sqlparse(form_dict)  <span class="comment"># sql拼接</span></span><br><span class="line">    df = pd.read_sql_query(sql, ENGINE)  <span class="comment"># 将sql语句结果读取至Pandas Dataframe</span></span><br><span class="line"></span><br><span class="line">    <span class="keyword">if</span> is_pivoted <span class="keyword">is</span> <span class="literal">True</span>:</span><br><span class="line">        dimension_selected = form_dict[<span class="string">&#x27;DIMENSION_select&#x27;</span>][<span class="number">0</span>]</span><br><span class="line">        <span class="keyword">if</span> dimension_selected[<span class="number">0</span>] == <span class="string">&#x27;[&#x27;</span>:</span><br><span class="line"></span><br><span class="line">            column = dimension_selected[<span class="number">1</span>:][:-<span class="number">1</span>]</span><br><span class="line">        <span class="keyword">else</span>:</span><br><span class="line">            column = dimension_selected</span><br><span class="line"></span><br><span class="line">        pivoted = pd.pivot_table(df,</span><br><span class="line">                                 values=<span class="string">&#x27;AMOUNT&#x27;</span>,  <span class="comment"># 数据透视汇总值为AMOUNT字段，一般保持不变</span></span><br><span class="line">                                 index=<span class="string">&#x27;DATE&#x27;</span>,  <span class="comment"># 数据透视行为DATE字段，一般保持不变</span></span><br><span class="line">                                 columns=column,  <span class="comment"># 数据透视列为前端选择的分析维度</span></span><br><span class="line">                                 aggfunc=np.<span class="built_in">sum</span>)  <span class="comment"># 数据透视汇总方式为求和，一般保持不变</span></span><br><span class="line">        <span class="keyword">if</span> pivoted.empty <span class="keyword">is</span> <span class="literal">False</span>:</span><br><span class="line">            pivoted.sort_values(by=pivoted.index[-<span class="number">1</span>], axis=<span class="number">1</span>, ascending=<span class="literal">False</span>, inplace=<span class="literal">True</span>)  <span class="comment"># 结果按照最后一个DATE表现排序</span></span><br><span class="line"></span><br><span class="line">        <span class="keyword">return</span> pivoted</span><br><span class="line">    <span class="keyword">else</span>:</span><br><span class="line">        <span class="keyword">return</span> df</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="keyword">def</span> <span class="title function_">query</span>(<span class="params">request</span>):</span><br><span class="line">    form_dict = <span class="built_in">dict</span>(six.iterlists(request.GET))</span><br><span class="line">    pivoted = get_df(form_dict)</span><br><span class="line"></span><br><span class="line">    <span class="comment"># KPI</span></span><br><span class="line">    kpi = get_kpi(pivoted)</span><br><span class="line"></span><br><span class="line">    table = ptable(pivoted)</span><br><span class="line">    table = table.to_html(formatters=build_formatters_by_col(table),  <span class="comment"># 逐列调整表格内数字格式</span></span><br><span class="line">                          classes=<span class="string">&#x27;ui selectable celled table&#x27;</span>,  <span class="comment"># 指定表格css class为Semantic UI主题</span></span><br><span class="line">                          table_id=<span class="string">&#x27;ptable&#x27;</span>  <span class="comment"># 指定表格id</span></span><br><span class="line">                          )</span><br><span class="line"></span><br><span class="line">    <span class="comment"># Pyecharts交互图表</span></span><br><span class="line">    bar_total_trend = json.loads(prepare_chart(pivoted, <span class="string">&#x27;bar_total_trend&#x27;</span>, form_dict))</span><br><span class="line"></span><br><span class="line">    <span class="comment"># Matplotlib静态图表</span></span><br><span class="line">    bubble_performance = prepare_chart(pivoted, <span class="string">&#x27;bubble_performance&#x27;</span>, form_dict)</span><br><span class="line">    context = &#123;</span><br><span class="line">        <span class="string">&quot;market_size&quot;</span>: kpi[<span class="string">&quot;market_size&quot;</span>],</span><br><span class="line">        <span class="string">&quot;market_gr&quot;</span>: kpi[<span class="string">&quot;market_gr&quot;</span>],</span><br><span class="line">        <span class="string">&quot;market_cagr&quot;</span>: kpi[<span class="string">&quot;market_cagr&quot;</span>],</span><br><span class="line">        <span class="string">&#x27;ptable&#x27;</span>: table,</span><br><span class="line">        <span class="string">&#x27;bar_total_trend&#x27;</span>: bar_total_trend,</span><br><span class="line">        <span class="string">&#x27;bubble_performance&#x27;</span>: bubble_performance</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="keyword">return</span> HttpResponse(json.dumps(context, ensure_ascii=<span class="literal">False</span>), content_type=<span class="string">&quot;application/json charset=utf-8&quot;</span>) <span class="comment"># 返回结果必须是json格式</span></span><br></pre></td></tr></table></figure><p>后端实现导出功能新建的export方法也可以和query方法一样在开头调用get_df：</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">try</span>:</span><br><span class="line">    <span class="keyword">from</span> io <span class="keyword">import</span> BytesIO <span class="keyword">as</span> IO <span class="comment"># for modern python</span></span><br><span class="line"><span class="keyword">except</span> ImportError:</span><br><span class="line">    <span class="keyword">from</span> io <span class="keyword">import</span> StringIO <span class="keyword">as</span> IO <span class="comment"># for legacy python</span></span><br><span class="line"><span class="keyword">import</span> datetime</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="keyword">def</span> <span class="title function_">export</span>(<span class="params">request, <span class="built_in">type</span></span>):</span><br><span class="line">    form_dict = <span class="built_in">dict</span>(six.iterlists(request.GET))</span><br><span class="line"></span><br><span class="line">    <span class="keyword">if</span> <span class="built_in">type</span> == <span class="string">&#x27;pivoted&#x27;</span>:</span><br><span class="line">        df = get_df(form_dict)  <span class="comment"># 透视后的数据</span></span><br><span class="line">    <span class="keyword">elif</span> <span class="built_in">type</span> == <span class="string">&#x27;raw&#x27;</span>:</span><br><span class="line">        df = get_df(form_dict, is_pivoted=<span class="literal">False</span>)  <span class="comment"># 原始数</span></span><br><span class="line"></span><br><span class="line">    excel_file = IO()</span><br><span class="line"></span><br><span class="line">    xlwriter = pd.ExcelWriter(excel_file, engine=<span class="string">&#x27;xlsxwriter&#x27;</span>)</span><br><span class="line"></span><br><span class="line">    df.to_excel(xlwriter, <span class="string">&#x27;data&#x27;</span>, index=<span class="literal">True</span>)</span><br><span class="line"></span><br><span class="line">    xlwriter.save()</span><br><span class="line">    xlwriter.close()</span><br><span class="line"></span><br><span class="line">    excel_file.seek(<span class="number">0</span>)</span><br><span class="line"></span><br><span class="line">    <span class="comment"># 设置浏览器mime类型</span></span><br><span class="line">    response = HttpResponse(excel_file.read(),</span><br><span class="line">                            content_type=<span class="string">&#x27;application/vnd.openxmlformats-officedocument.spreadsheetml.sheet&#x27;</span>)</span><br><span class="line"></span><br><span class="line">    <span class="comment"># 设置文件名</span></span><br><span class="line">    now = datetime.datetime.now().strftime(<span class="string">&quot;%Y%m%d%H%M%S&quot;</span>)  <span class="comment"># 当前精确时间不会重复，适合用来命名默认导出文件</span></span><br><span class="line">    response[<span class="string">&#x27;Content-Disposition&#x27;</span>] = <span class="string">&#x27;attachment; filename=&#x27;</span> + now + <span class="string">&#x27;.xlsx&#x27;</span></span><br><span class="line">    <span class="keyword">return</span> response</span><br></pre></td></tr></table></figure><p>实现导出功能的Python写法五花八门，条条大路通罗马。我们这次代码的特点是使用了pandas的df.to_excel方法，需要加载xlsxwriter这个库。有些场景的导出方法则可能需要其他一些处理Excel的库。这段代码最需要注意的是最后设置浏览器mime类型和文件名部分的写法。</p><p>相应地，在url.py里再增加一个url pattern:</p><figure class="highlight text"><table><tr><td class="code"><pre><span class="line">urlpatterns = [</span><br><span class="line">    ...</span><br><span class="line">    path(r&#x27;export/&lt;str:type&gt;&#x27;, views.export, name=&#x27;export&#x27;),</span><br><span class="line">]</span><br></pre></td></tr></table></figure><p>前端因为也不是一个AJAX URL就返回所有数据了，也产生了代码复用的需求，我们把filter.html下面这段获取表单选择结果的JS代码独立出来：</p><figure class="highlight js"><table><tr><td class="code"><pre><span class="line">&lt;script&gt;</span><br><span class="line">    <span class="keyword">function</span> <span class="title function_">getForm</span>(<span class="params"></span>)&#123;</span><br><span class="line">        <span class="comment">// 获取单选下拉框的值</span></span><br><span class="line">        <span class="keyword">var</span> form_data = &#123;</span><br><span class="line">            <span class="string">&quot;DIMENSION_select&quot;</span>: $(<span class="string">&quot;#DIMENSION_select&quot;</span>).<span class="title function_">val</span>(),</span><br><span class="line">            <span class="string">&quot;PERIOD_select&quot;</span>: $(<span class="string">&quot;#PERIOD_select&quot;</span>).<span class="title function_">val</span>(),</span><br><span class="line">            <span class="string">&quot;UNIT_select&quot;</span>: $(<span class="string">&quot;#UNIT_select&quot;</span>).<span class="title function_">val</span>(),</span><br><span class="line">        &#125;;</span><br><span class="line"></span><br><span class="line">        <span class="comment">// 获取多选下拉框的值</span></span><br><span class="line">        <span class="keyword">var</span> dict = &#123;&amp;#<span class="number">123</span>; mselect_dict|safe &amp;#<span class="number">125</span>;&#125;;</span><br><span class="line">        <span class="keyword">for</span> (key <span class="keyword">in</span> dict) &#123;</span><br><span class="line">            <span class="keyword">var</span> form_name = dict[key][<span class="string">&#x27;select&#x27;</span>] + <span class="string">&quot;_select&quot;</span>;</span><br><span class="line">            jquery_selector_id = <span class="string">&quot;[id=&#x27;&quot;</span> + form_name + <span class="string">&quot;&#x27;]&quot;</span>;<span class="comment">//因为我们的部分多选框id有空格，要用这种写法</span></span><br><span class="line">            form_data[form_name] = $(jquery_selector_id).<span class="title function_">val</span>();</span><br><span class="line">        &#125;</span><br><span class="line"></span><br><span class="line">        <span class="keyword">return</span> form_data</span><br><span class="line">    &#125;</span><br><span class="line">&lt;/script&gt;</span><br></pre></td></tr></table></figure><p>在display.html里新建两个导出按钮，并写上相应的鼠标点击函数，注意这里的传参用了+ ‘?’ + $.param()的写法：</p><figure class="highlight js"><table><tr><td class="code"><pre><span class="line">&lt;div <span class="keyword">class</span>=<span class="string">&quot;ui pointing secondary menu&quot;</span>&gt;</span><br><span class="line">    ...</span><br><span class="line">    &lt;a <span class="keyword">class</span>=<span class="string">&quot;item&quot;</span> data-tab=<span class="string">&quot;export&quot;</span>&gt;<span class="language-xml"><span class="tag">&lt;<span class="name">i</span> <span class="attr">class</span>=<span class="string">&quot;download icon&quot;</span>&gt;</span><span class="tag">&lt;/<span class="name">i</span>&gt;</span></span>导出数据&lt;/a&gt;</span><br><span class="line">&lt;/div&gt;</span><br><span class="line">...</span><br><span class="line">&lt;div <span class="keyword">class</span>=<span class="string">&quot;ui tab segment&quot;</span> data-tab=<span class="string">&quot;export&quot;</span>&gt;</span><br><span class="line">    <span class="language-xml"><span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;ui buttons&quot;</span>&gt;</span></span></span><br><span class="line"><span class="language-xml">        <span class="tag">&lt;<span class="name">input</span> <span class="attr">class</span>=<span class="string">&quot;ui blue button&quot;</span> <span class="attr">type</span>=<span class="string">&#x27;button&#x27;</span> <span class="attr">id</span>=<span class="string">&#x27;export_pivot&#x27;</span> <span class="attr">value</span>=<span class="string">&quot;导出整理后时间序列数据&quot;</span>/&gt;</span></span></span><br><span class="line"><span class="language-xml">    <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span></span><br><span class="line">    <span class="language-xml"><span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;ui buttons&quot;</span>&gt;</span></span></span><br><span class="line"><span class="language-xml">        <span class="tag">&lt;<span class="name">input</span> <span class="attr">class</span>=<span class="string">&quot;ui blue button&quot;</span> <span class="attr">type</span>=<span class="string">&#x27;button&#x27;</span> <span class="attr">id</span>=<span class="string">&#x27;export_raw&#x27;</span> <span class="attr">value</span>=<span class="string">&quot;导出原始数据&quot;</span>/&gt;</span></span></span><br><span class="line"><span class="language-xml">    <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span></span><br><span class="line">&lt;/div&gt;</span><br><span class="line"></span><br><span class="line"><span class="language-xml"><span class="tag">&lt;<span class="name">script</span>&gt;</span><span class="language-javascript"></span></span></span><br><span class="line"><span class="language-javascript"><span class="language-xml">    $(<span class="string">&quot;#export_pivot&quot;</span>).<span class="title function_">click</span>(<span class="keyword">function</span>(<span class="params"></span>)&#123;</span></span></span><br><span class="line"><span class="language-javascript"><span class="language-xml">        <span class="keyword">var</span> form_data = <span class="title function_">getForm</span>();</span></span></span><br><span class="line"><span class="language-javascript"><span class="language-xml"></span></span></span><br><span class="line"><span class="language-javascript"><span class="language-xml">        <span class="keyword">var</span> downloadUrl = <span class="string">&#x27;&#123;&amp;#37; url &#x27;</span><span class="attr">chpa</span>:<span class="keyword">export</span><span class="string">&#x27; &#x27;</span>pivoted<span class="string">&#x27; &amp;#37;&#125;&#x27;</span>+ <span class="string">&#x27;?&#x27;</span> + $.<span class="title function_">param</span>(form_data, <span class="literal">true</span>);</span></span></span><br><span class="line"><span class="language-javascript"><span class="language-xml">        <span class="variable language_">window</span>.<span class="property">location</span>.<span class="property">href</span> = downloadUrl;</span></span></span><br><span class="line"><span class="language-javascript"><span class="language-xml">    &#125;);</span></span></span><br><span class="line"><span class="language-javascript"><span class="language-xml"></span></span></span><br><span class="line"><span class="language-javascript"><span class="language-xml">    $(<span class="string">&quot;#export_raw&quot;</span>).<span class="title function_">click</span>(<span class="keyword">function</span>(<span class="params"></span>)&#123;</span></span></span><br><span class="line"><span class="language-javascript"><span class="language-xml">        <span class="keyword">var</span> form_data = <span class="title function_">getForm</span>();</span></span></span><br><span class="line"><span class="language-javascript"><span class="language-xml"></span></span></span><br><span class="line"><span class="language-javascript"><span class="language-xml">        <span class="keyword">var</span> downloadUrl = <span class="string">&#x27;&#123;&amp;#37; url &#x27;</span><span class="attr">chpa</span>:<span class="keyword">export</span><span class="string">&#x27; &#x27;</span>raw<span class="string">&#x27; &amp;#37;&#125;&#x27;</span>+ <span class="string">&#x27;?&#x27;</span> + $.<span class="title function_">param</span>(form_data, <span class="literal">true</span>);</span></span></span><br><span class="line"><span class="language-javascript"><span class="language-xml">        <span class="variable language_">window</span>.<span class="property">location</span>.<span class="property">href</span> = downloadUrl;</span></span></span><br><span class="line"><span class="language-javascript"><span class="language-xml">    &#125;)</span></span></span><br><span class="line"><span class="language-javascript"><span class="language-xml"></span><span class="tag">&lt;/<span class="name">script</span>&gt;</span></span></span><br></pre></td></tr></table></figure><p>最后需要注意这里的超级大坑，还记得第五章传参时表单多选框[]结尾的问题吗，这里因为不是AJAX传参，这时的多选框又不以[]结尾传参了，这种前后的不一致可能会导致后端混乱。我们需要在后端涉及到的地方顾忌这个问题，比如SQL语句拼接的函数中：</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">def</span> <span class="title function_">sqlparse</span>(<span class="params">context</span>):</span><br><span class="line">    <span class="built_in">print</span>(context)</span><br><span class="line">    sql = <span class="string">&quot;Select * from %s Where PERIOD = &#x27;%s&#x27; And UNIT = &#x27;%s&#x27;&quot;</span> % \</span><br><span class="line">          (DB_TABLE, context[<span class="string">&#x27;PERIOD_select&#x27;</span>][<span class="number">0</span>], context[<span class="string">&#x27;UNIT_select&#x27;</span>][<span class="number">0</span>])  <span class="comment"># 先处理单选部分</span></span><br><span class="line"></span><br><span class="line">    <span class="comment"># 下面循环处理多选部分</span></span><br><span class="line">    <span class="keyword">for</span> k, v <span class="keyword">in</span> context.items():</span><br><span class="line">        <span class="keyword">if</span> k <span class="keyword">not</span> <span class="keyword">in</span> [<span class="string">&#x27;csrfmiddlewaretoken&#x27;</span>, <span class="string">&#x27;DIMENSION_select&#x27;</span>, <span class="string">&#x27;PERIOD_select&#x27;</span>, <span class="string">&#x27;UNIT_select&#x27;</span>]:</span><br><span class="line">            <span class="keyword">if</span> k[-<span class="number">2</span>:] == <span class="string">&#x27;[]&#x27;</span>:</span><br><span class="line">                field_name = k[:-<span class="number">9</span>]  <span class="comment"># 如果键以[]结尾，删除_select[]取原字段名</span></span><br><span class="line">            <span class="keyword">else</span>:</span><br><span class="line">                field_name = k[:-<span class="number">7</span>]  <span class="comment"># 如果键不以[]结尾，删除_select取原字段名</span></span><br><span class="line">            selected = v  <span class="comment"># 选择项</span></span><br><span class="line">            sql = sql_extent(sql, field_name, selected)  <span class="comment">#未来可以通过进一步拼接字符串动态扩展sql语句</span></span><br><span class="line">    <span class="keyword">return</span> sql</span><br></pre></td></tr></table></figure><p>完成。<br><img src="/images/python-djangosqlpand/v2-b5db0877f6596c5f56a61f8d9da9c494_1440w.webp"></p><p>导出按钮可以视为一种可视化输出与其他可视化布局是并列关系<br><img src="/images/python-djangosqlpand/v2-1058c901e2c3895c62cf52e20a95ceb6_1440w.webp"></p><p>第一种导出形式，透视为时间序列后的数据<br><img src="/images/python-djangosqlpand/v2-410173def083f62662c0b11066b37520_1440w.webp"></p><p>第二种导出形式，纯原始数</p>]]>
    </content>
    <id>https://blog.malu.tech/python-django-sql-pandas-pyecharts-data-analysis-platform-11/</id>
    <link href="https://blog.malu.tech/python-django-sql-pandas-pyecharts-data-analysis-platform-11/"/>
    <published>2024-03-10T16:00:00.000Z</published>
    <summary>Python Django+SQL+Pandas+Pyecharts自建在线数据分析平台系列文章第十一篇：导出数据至Excel功能</summary>
    <title>Python Django+SQL+Pandas+Pyecharts自建在线数据分析平台（十一）</title>
    <updated>2024-03-10T16:00:00.000Z</updated>
  </entry>
  <entry>
    <author>
      <name>Road</name>
    </author>
    <category term="Python" scheme="https://blog.malu.tech/categories/Python/"/>
    <category term="Django" scheme="https://blog.malu.tech/categories/Python/Django/"/>
    <category term="数据分析" scheme="https://blog.malu.tech/categories/Python/Django/%E6%95%B0%E6%8D%AE%E5%88%86%E6%9E%90/"/>
    <category term="Python" scheme="https://blog.malu.tech/tags/Python/"/>
    <category term="Django" scheme="https://blog.malu.tech/tags/Django/"/>
    <category term="SQL" scheme="https://blog.malu.tech/tags/SQL/"/>
    <category term="Pandas" scheme="https://blog.malu.tech/tags/Pandas/"/>
    <category term="Pyecharts" scheme="https://blog.malu.tech/tags/Pyecharts/"/>
    <category term="数据分析平台" scheme="https://blog.malu.tech/tags/%E6%95%B0%E6%8D%AE%E5%88%86%E6%9E%90%E5%B9%B3%E5%8F%B0/"/>
    <content>
      <![CDATA[<blockquote><p><strong>文章来源</strong>：<a href="https://zhuanlan.zhihu.com/p/142490087">Python Django+SQL+Pandas+Pyecharts自建在线数据分析平台（一）</a><br><strong>作者</strong>：<a href="https://www.zhihu.com/people/chen-cheng-76-40">ccpic</a><br><img src="/images/python-djangosqlpand/v2-3bbd9dc618a329f4aceeba2aef0632fb_1440w.webp"><br><strong>感谢</strong>：感谢作者 ccpic 分享的优质内容，本网页主要用于学习知识的存档备份，欢迎点击原网页支持作者。</p></blockquote><p>（一）需求分析&amp;技术实现</p><p>（二）初步搭建Django环境</p><p>（三）页面布局&amp;Django模板</p><p>（四）SQL+Pandas初步处理数据</p><p>（五）前端表单交互</p><p>（六）Ajax异步传参与加载</p><p>（七）前端数据格式的处理</p><p>（八）DataTables接管前端表格</p><p>（九）Pyecharts实现交互图表</p><h2 id="（十）静态图表的展示"><a href="#（十）静态图表的展示" class="headerlink" title="（十）静态图表的展示"></a>（十）静态图表的展示</h2><p>（十一）“导出数据至Excel”功能</p><p>（十二）添加和配置缓存</p><p>（十三）用户登录系统</p><p>（十四）部署Django至生产环境</p><p>在数据可视化的工作中，静态图表是一个非常容易被忽视的形式。但在信息化程度较低的传统行业中，静态图表却是必不可缺的存在，主要是因为：</p><blockquote><p>让老板们自己操作BI是天方夜谭</p></blockquote><p>好吧，即使抛开老板们的主观能动性和BI的操作难度，我们也要有觉悟——PPT还是大部分行业工作中汇报与沟通的最常用媒介。而以Matplotlib为代表的的静态图表在某些方面还是更加契合这个媒介：</p><ul><li><strong>Echarts, Highcharts等在线交互图表注定会因为“交互”的特性而不将一些信息置于表面（如鼠标hover才显示的tooltip），截图或保存图片无法呈现。</strong></li><li><strong>在线交互图表靠截图或保存图片展示的DPI远低于手动绘制的图表。</strong></li><li><strong>Matplotlib以及衍生的Seaborn等静态图表库往往具有高度可定制化的特性，对细节的展示度更优。</strong></li><li><strong>Matplotlib以及衍生的Seaborn等静态图表库与统计学方法的展示结合更容易。</strong></li></ul><p>以下面Matplotlib绘制的气泡图为例，展示了治疗糖尿病的胰岛素市场所有产品的销售额规模和净增长，中间有一个线性回归拟合的趋势线及CI和PI（回归带上方的是净增长显著高于预期，下方则反之，落在带内的则是统计学上不显著）。其中还有一些定制化的细节呈现方式，如只显示top30产品的名字；文字标签相互之间不堆叠等。</p><p>现有的在线图表方案几乎很难呈现这样的效果，但这却是Matplotlib或者Seaborn的拿手好戏。</p><p>所以本章的目标是用我们在建的Django系统也能动态展示上方这样的Matplotlib静态图表。<br><img src="/images/python-djangosqlpand/v2-7ce174eee89f10eeaa5fc351aab2e69a_1440w.webp"></p><p>在上章我们创建的charts.py中新建一个mpl_bubble方法，绘制气泡图：</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">import</span> numpy <span class="keyword">as</span> np</span><br><span class="line"><span class="keyword">import</span> matplotlib.pyplot <span class="keyword">as</span> plt</span><br><span class="line"><span class="keyword">import</span> matplotlib.font_manager <span class="keyword">as</span> fm</span><br><span class="line"><span class="keyword">import</span> matplotlib <span class="keyword">as</span> mpl</span><br><span class="line"><span class="keyword">from</span> matplotlib.ticker <span class="keyword">import</span> FuncFormatter</span><br><span class="line"><span class="keyword">from</span> adjustText <span class="keyword">import</span> adjust_text</span><br><span class="line"><span class="keyword">from</span> io <span class="keyword">import</span> BytesIO</span><br><span class="line"><span class="keyword">import</span> base64</span><br><span class="line"><span class="keyword">import</span> scipy.stats <span class="keyword">as</span> stats</span><br><span class="line"></span><br><span class="line">myfont = fm.FontProperties(fname=<span class="string">&#x27;C:/Windows/Fonts/msyh.ttc&#x27;</span>)</span><br><span class="line"></span><br><span class="line"><span class="keyword">def</span> <span class="title function_">mpl_bubble</span>(<span class="params">x, y, z, labels, title, x_title, y_title,</span></span><br><span class="line"><span class="params">               x_fmt=<span class="string">&#x27;&#123;:.0&amp;#37;&#125;&#x27;</span>, y_fmt=<span class="string">&#x27;&#123;:+.0&amp;#37;&#125;&#x27;</span>,</span></span><br><span class="line"><span class="params">               y_avg_line=<span class="literal">False</span>, y_avg_value=<span class="literal">None</span>, y_avg_label=<span class="string">&#x27;&#x27;</span>,</span></span><br><span class="line"><span class="params">               x_avg_line=<span class="literal">False</span>, x_avg_value=<span class="literal">None</span>, x_avg_label=<span class="string">&#x27;&#x27;</span>,</span></span><br><span class="line"><span class="params">               x_max=<span class="literal">None</span>, x_min=<span class="literal">None</span>, y_max=<span class="literal">None</span>, y_min=<span class="literal">None</span>,</span></span><br><span class="line"><span class="params">               show_label=<span class="literal">True</span>, label_limit=<span class="number">15</span>,</span></span><br><span class="line"><span class="params">               z_scale=<span class="number">1</span>, color_scheme=<span class="string">&#x27;随机颜色方案&#x27;</span>, color_list=<span class="literal">None</span></span>):</span><br><span class="line"></span><br><span class="line">    z = [x * z_scale <span class="keyword">for</span> x <span class="keyword">in</span> z]  <span class="comment"># 气泡大小系数</span></span><br><span class="line"></span><br><span class="line">    fig, ax = plt.subplots()  <span class="comment"># 准备画布和轴</span></span><br><span class="line">    fig.set_size_inches(<span class="number">15</span>, <span class="number">10</span>)  <span class="comment"># 画布尺寸</span></span><br><span class="line"></span><br><span class="line">    <span class="comment"># 手动强制xy轴最小值/最大值</span></span><br><span class="line">    <span class="keyword">if</span> x_min <span class="keyword">is</span> <span class="keyword">not</span> <span class="literal">None</span> <span class="keyword">and</span> x_min &gt; <span class="built_in">min</span>(x):</span><br><span class="line">        ax.set_xlim(xmin=x_min)</span><br><span class="line">    <span class="keyword">if</span> x_max <span class="keyword">is</span> <span class="keyword">not</span> <span class="literal">None</span> <span class="keyword">and</span> x_max &lt; <span class="built_in">max</span>(x):</span><br><span class="line">        ax.set_xlim(xmax=x_max)</span><br><span class="line">    <span class="keyword">if</span> y_min <span class="keyword">is</span> <span class="keyword">not</span> <span class="literal">None</span> <span class="keyword">and</span> y_min &gt; <span class="built_in">min</span>(y):</span><br><span class="line">        ax.set_ylim(ymin=y_min)</span><br><span class="line">    <span class="keyword">if</span> y_max <span class="keyword">is</span> <span class="keyword">not</span> <span class="literal">None</span> <span class="keyword">and</span> y_max &lt; <span class="built_in">max</span>(y):</span><br><span class="line">        ax.set_ylim(ymax=y_max)</span><br><span class="line"></span><br><span class="line">    <span class="comment"># 确定颜色方案</span></span><br><span class="line">    <span class="keyword">if</span> color_scheme == <span class="string">&#x27;随机颜色方案&#x27;</span> <span class="keyword">or</span> color_scheme <span class="keyword">is</span> <span class="literal">None</span>:</span><br><span class="line">        cmap = mpl.colors.ListedColormap(np.random.rand(<span class="number">256</span>, <span class="number">3</span>))</span><br><span class="line">        colors = <span class="built_in">iter</span>(cmap(np.linspace(<span class="number">0</span>, <span class="number">1</span>, <span class="built_in">len</span>(x))))</span><br><span class="line">    <span class="keyword">else</span>:</span><br><span class="line">        <span class="keyword">if</span> <span class="built_in">len</span>(x) &lt;= <span class="built_in">len</span>(color_list):</span><br><span class="line">            colors = color_list[:<span class="built_in">len</span>(x)]</span><br><span class="line">        <span class="keyword">else</span>:</span><br><span class="line">            colors = []</span><br><span class="line">            <span class="keyword">for</span> i <span class="keyword">in</span> <span class="built_in">range</span>(<span class="built_in">len</span>(x)):</span><br><span class="line">                colors.append(color_list[i % <span class="built_in">len</span>(color_list)])</span><br><span class="line">        colors = <span class="built_in">iter</span>(colors)</span><br><span class="line"></span><br><span class="line">    <span class="comment"># 绘制气泡</span></span><br><span class="line">    <span class="keyword">for</span> i <span class="keyword">in</span> <span class="built_in">range</span>(<span class="built_in">len</span>(x)):</span><br><span class="line">        ax.scatter(x[i], y[i], z[i], color=<span class="built_in">next</span>(colors), alpha=<span class="number">0.6</span>, edgecolors=<span class="string">&quot;black&quot;</span>)</span><br><span class="line"></span><br><span class="line">    <span class="comment"># 添加系列标签，用adjust_text包保证标签互不重叠</span></span><br><span class="line">    <span class="keyword">if</span> show_label <span class="keyword">is</span> <span class="literal">True</span>:</span><br><span class="line">        texts = [plt.text(x[i], y[i], labels[i],</span><br><span class="line">                          ha=<span class="string">&#x27;center&#x27;</span>, va=<span class="string">&#x27;center&#x27;</span>, multialignment=<span class="string">&#x27;center&#x27;</span>, fontproperties=myfont, fontsize=<span class="number">10</span>) <span class="keyword">for</span></span><br><span class="line">                 i</span><br><span class="line">                 <span class="keyword">in</span> <span class="built_in">range</span>(<span class="built_in">len</span>(labels[:label_limit]))]</span><br><span class="line">        adjust_text(texts, force_text=<span class="number">0.5</span>, arrowprops=<span class="built_in">dict</span>(arrowstyle=<span class="string">&#x27;-&gt;&#x27;</span>, color=<span class="string">&#x27;black&#x27;</span>))</span><br><span class="line"></span><br><span class="line">    <span class="comment"># 添加分隔线（均值，中位数，0等）</span></span><br><span class="line">    <span class="keyword">if</span> y_avg_line <span class="keyword">is</span> <span class="literal">True</span>:</span><br><span class="line">        ax.axhline(y_avg_value, linestyle=<span class="string">&#x27;--&#x27;</span>, linewidth=<span class="number">1</span>, color=<span class="string">&#x27;grey&#x27;</span>)</span><br><span class="line">        plt.text(ax.get_xlim()[<span class="number">1</span>], y_avg_value, y_avg_label, ha=<span class="string">&#x27;left&#x27;</span>, va=<span class="string">&#x27;center&#x27;</span>, color=<span class="string">&#x27;r&#x27;</span>,</span><br><span class="line">                 multialignment=<span class="string">&#x27;center&#x27;</span>,</span><br><span class="line">                 fontproperties=myfont, fontsize=<span class="number">10</span>)</span><br><span class="line">    <span class="keyword">if</span> x_avg_line <span class="keyword">is</span> <span class="literal">True</span>:</span><br><span class="line">        ax.axvline(x_avg_value, linestyle=<span class="string">&#x27;--&#x27;</span>, linewidth=<span class="number">1</span>, color=<span class="string">&#x27;grey&#x27;</span>)</span><br><span class="line">        plt.text(x_avg_value, ax.get_ylim()[<span class="number">1</span>], x_avg_label, ha=<span class="string">&#x27;left&#x27;</span>, va=<span class="string">&#x27;top&#x27;</span>,</span><br><span class="line">                 color=<span class="string">&#x27;r&#x27;</span>, multialignment=<span class="string">&#x27;center&#x27;</span>, fontproperties=myfont, fontsize=<span class="number">10</span>)</span><br><span class="line"></span><br><span class="line">    <span class="comment"># 设置轴标签格式</span></span><br><span class="line">    ax.xaxis.set_major_formatter(FuncFormatter(<span class="keyword">lambda</span> y, _: x_fmt.<span class="built_in">format</span>(y)))</span><br><span class="line">    ax.yaxis.set_major_formatter(FuncFormatter(<span class="keyword">lambda</span> y, _: y_fmt.<span class="built_in">format</span>(y)))</span><br><span class="line"></span><br><span class="line">    <span class="comment"># 添加图表标题和轴标题</span></span><br><span class="line">    plt.title(title, fontproperties=myfont)</span><br><span class="line">    plt.xlabel(x_title, fontproperties=myfont, fontsize=<span class="number">12</span>)</span><br><span class="line">    plt.ylabel(y_title, fontproperties=myfont, fontsize=<span class="number">12</span>)</span><br><span class="line"></span><br><span class="line">    <span class="string">&quot;&quot;&quot;以下部分绘制回归拟合曲线及CI和PI</span></span><br><span class="line"><span class="string">    参考</span></span><br><span class="line"><span class="string">    http://nbviewer.ipython.org/github/demotu/BMC/blob/master/notebooks/CurveFitting.ipynb</span></span><br><span class="line"><span class="string">    https://stackoverflow.com/questions/27164114/show-confidence-limits-and-prediction-limits-in-scatter-plot</span></span><br><span class="line"><span class="string">    &quot;&quot;&quot;</span></span><br><span class="line"></span><br><span class="line">    n = y.size  <span class="comment"># 观察例数</span></span><br><span class="line">    <span class="keyword">if</span> n &gt; <span class="number">2</span>:  <span class="comment"># 数据点必须大于cov矩阵的scale</span></span><br><span class="line">        p, cov = np.polyfit(x, y, <span class="number">1</span>, cov=<span class="literal">True</span>) <span class="comment"># 简单线性回归返回parameter和covariance</span></span><br><span class="line">        poly1d_fn = np.poly1d(p)  <span class="comment"># 拟合方程</span></span><br><span class="line">        y_model = poly1d_fn(x)  <span class="comment"># 拟合的y值</span></span><br><span class="line">        m = p.size  <span class="comment"># 参数个数</span></span><br><span class="line"></span><br><span class="line">        dof = n - m  <span class="comment"># degrees of freedom</span></span><br><span class="line">        t = stats.t.ppf(<span class="number">0.975</span>, dof)  <span class="comment"># 显著性检验t值</span></span><br><span class="line"></span><br><span class="line">        <span class="comment"># 拟合结果绘图</span></span><br><span class="line">        ax.plot(x, y_model, <span class="string">&quot;-&quot;</span>, color=<span class="string">&quot;0.1&quot;</span>, linewidth=<span class="number">1.5</span>, alpha=<span class="number">0.5</span>, label=<span class="string">&quot;Fit&quot;</span>)</span><br><span class="line"></span><br><span class="line">        <span class="comment"># 误差估计</span></span><br><span class="line">        resid = y - y_model  <span class="comment"># 残差</span></span><br><span class="line">        s_err = np.sqrt(np.<span class="built_in">sum</span>(resid ** <span class="number">2</span>) / dof)  <span class="comment"># 标准误差</span></span><br><span class="line"></span><br><span class="line">        <span class="comment"># 拟合CI和PI</span></span><br><span class="line">        x2 = np.linspace(np.<span class="built_in">min</span>(x), np.<span class="built_in">max</span>(x), <span class="number">100</span>)</span><br><span class="line">        y2 = poly1d_fn(x2)</span><br><span class="line"></span><br><span class="line">        <span class="comment"># CI计算和绘图</span></span><br><span class="line">        ci = t * s_err * np.sqrt(<span class="number">1</span> / n + (x2 - np.mean(x)) ** <span class="number">2</span> / np.<span class="built_in">sum</span>((x - np.mean(x)) ** <span class="number">2</span>))</span><br><span class="line">        ax.fill_between(x2, y2 + ci, y2 - ci, color=<span class="string">&quot;#b9cfe7&quot;</span>, edgecolor=<span class="string">&quot;&quot;</span>, alpha=<span class="number">0.5</span>)</span><br><span class="line"></span><br><span class="line">        <span class="comment"># Pi计算和绘图</span></span><br><span class="line">        pi = t * s_err * np.sqrt(<span class="number">1</span> + <span class="number">1</span> / n + (x2 - np.mean(x)) ** <span class="number">2</span> / np.<span class="built_in">sum</span>((x - np.mean(x)) ** <span class="number">2</span>))</span><br><span class="line">        ax.fill_between(x2, y2 + pi, y2 - pi, color=<span class="string">&quot;None&quot;</span>, linestyle=<span class="string">&quot;--&quot;</span>)</span><br><span class="line">        ax.plot(x2, y2 - pi, <span class="string">&quot;--&quot;</span>, color=<span class="string">&quot;0.5&quot;</span>, label=<span class="string">&quot;95% Prediction Limits&quot;</span>)</span><br><span class="line">        ax.plot(x2, y2 + pi, <span class="string">&quot;--&quot;</span>, color=<span class="string">&quot;0.5&quot;</span>)</span><br><span class="line"></span><br><span class="line">    <span class="comment"># 保存到字符串</span></span><br><span class="line">    sio = BytesIO()</span><br><span class="line">    plt.savefig(sio, <span class="built_in">format</span>=<span class="string">&#x27;png&#x27;</span>, bbox_inches=<span class="string">&#x27;tight&#x27;</span>, transparent=<span class="literal">True</span>, dpi=<span class="number">600</span>)</span><br><span class="line">    data = base64.encodebytes(sio.getvalue()).decode()  <span class="comment"># 解码为base64编码的png图片数据</span></span><br><span class="line">    src = <span class="string">&#x27;data:image/png;base64,&#x27;</span> + <span class="built_in">str</span>(data)  <span class="comment"># 增加Data URI scheme</span></span><br><span class="line"></span><br><span class="line">    <span class="comment"># 关闭绘图进程</span></span><br><span class="line">    plt.clf()</span><br><span class="line">    plt.cla()</span><br><span class="line">    plt.close()</span><br><span class="line"></span><br><span class="line">    <span class="keyword">return</span> src</span><br></pre></td></tr></table></figure><p>上方代码比较复杂，不要被吓着了，其实绝大部分都是为了完成Matplotlib的绘图结果。本章主要侧重于Django展示图片的方法，Matplotlib的部分不做讲解，毕竟那是另外一个超级大坑，能又写一个系列教程。</p><p>所以只需要特别注意下面这个片段，其他部分可以替换成你自己的matplotlib代码：</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">from</span> io <span class="keyword">import</span> BytesIO</span><br><span class="line"><span class="keyword">import</span> base64</span><br><span class="line">...</span><br><span class="line"></span><br><span class="line"><span class="keyword">def</span> <span class="title function_">mpl_bubble</span>(<span class="params">x, y, z, labels, title, x_title, y_title,</span></span><br><span class="line"><span class="params">               ...</span>):</span><br><span class="line"></span><br><span class="line">    ...</span><br><span class="line"></span><br><span class="line">    <span class="comment"># 保存到字符串</span></span><br><span class="line">    sio = BytesIO()</span><br><span class="line">    plt.savefig(sio, <span class="built_in">format</span>=<span class="string">&#x27;png&#x27;</span>, bbox_inches=<span class="string">&#x27;tight&#x27;</span>, transparent=<span class="literal">True</span>, dpi=<span class="number">600</span>)</span><br><span class="line">    data = base64.encodebytes(sio.getvalue()).decode()  <span class="comment"># 解码为base64编码的png图片数据</span></span><br><span class="line">    src = <span class="string">&#x27;data:image/png;base64,&#x27;</span> + <span class="built_in">str</span>(data)  <span class="comment"># 增加Data URI scheme</span></span><br><span class="line"></span><br><span class="line">    ...</span><br><span class="line"></span><br><span class="line">    <span class="keyword">return</span> src</span><br></pre></td></tr></table></figure><p>这段代码是将上方大量代码绘制的Matplotlib图片**保存为base64编码的字符串，再在头部加上Data URI scheme。**这步可以说是浏览器展示静态图片的核心步骤。</p><p>我们再扩展上一章views.py中的prepare_chart方法，增加一类”bubble_performance”生成数据并传入到之前准备好的绘图方法中。注意和Pyehcarts不一样，这里直接取得返回的值就好了，不需要.dump_options()：</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line">D_TRANS = &#123;</span><br><span class="line">            <span class="string">&#x27;MAT&#x27;</span>: <span class="string">&#x27;滚动年&#x27;</span>,</span><br><span class="line">            <span class="string">&#x27;QTR&#x27;</span>: <span class="string">&#x27;季度&#x27;</span>,</span><br><span class="line">            <span class="string">&#x27;Value&#x27;</span>: <span class="string">&#x27;金额&#x27;</span>,</span><br><span class="line">            <span class="string">&#x27;Volume&#x27;</span>: <span class="string">&#x27;盒数&#x27;</span>,</span><br><span class="line">            <span class="string">&#x27;Volume (Counting Unit)&#x27;</span>: <span class="string">&#x27;最小制剂单位数&#x27;</span>,</span><br><span class="line">            <span class="string">&#x27;滚动年&#x27;</span>: <span class="string">&#x27;MAT&#x27;</span>,</span><br><span class="line">            <span class="string">&#x27;季度&#x27;</span>: <span class="string">&#x27;QTR&#x27;</span>,</span><br><span class="line">            <span class="string">&#x27;金额&#x27;</span>: <span class="string">&#x27;Value&#x27;</span>,</span><br><span class="line">            <span class="string">&#x27;盒数&#x27;</span>: <span class="string">&#x27;Volume&#x27;</span>,</span><br><span class="line">            <span class="string">&#x27;最小制剂单位数&#x27;</span>: <span class="string">&#x27;Volume (Counting Unit)&#x27;</span></span><br><span class="line">           &#125;</span><br><span class="line"></span><br><span class="line"><span class="keyword">def</span> <span class="title function_">prepare_chart</span>(<span class="params">df,  <span class="comment"># 输入经过pivoted方法透视过的df，不是原始df</span></span></span><br><span class="line"><span class="params">                  chart_type,  <span class="comment"># 图表类型字符串，人为设置，根据图表类型不同做不同的Pandas数据处理，及生成不同的Pyechart对象</span></span></span><br><span class="line"><span class="params">                  form_dict,  <span class="comment"># 前端表单字典，用来获得一些变量作为图表的标签如单位</span></span></span><br><span class="line"><span class="params">                  </span>):</span><br><span class="line">    label = D_TRANS[form_dict[<span class="string">&#x27;PERIOD_select&#x27;</span>][<span class="number">0</span>]] + D_TRANS[form_dict[<span class="string">&#x27;UNIT_select&#x27;</span>][<span class="number">0</span>]]</span><br><span class="line"></span><br><span class="line">    <span class="keyword">if</span> chart_type == <span class="string">&#x27;bar_total_trend&#x27;</span>:</span><br><span class="line">        df_abs = df.<span class="built_in">sum</span>(axis=<span class="number">1</span>)  <span class="comment"># Pandas列汇总，返回一个N行1列的series，每行是一个date的市场综合</span></span><br><span class="line">        df_abs.index = df_abs.index.strftime(<span class="string">&quot;%Y-%m&quot;</span>)  <span class="comment"># 行索引日期数据变成2020-06的形式</span></span><br><span class="line">        df_abs = df_abs.to_frame()  <span class="comment"># series转换成df</span></span><br><span class="line">        df_abs.columns = [label]  <span class="comment"># 用一些设置变量为系列命名，准备作为图表标签</span></span><br><span class="line">        df_gr = df_abs.pct_change(periods=<span class="number">4</span>)  <span class="comment"># 获取同比增长率</span></span><br><span class="line">        df_gr.dropna(how=<span class="string">&#x27;all&#x27;</span>, inplace=<span class="literal">True</span>)  <span class="comment"># 删除没有同比增长率的行，也就是时间序列数据的最前面几行，他们没有同比</span></span><br><span class="line">        df_gr.replace([np.inf, -np.inf, np.nan], <span class="string">&#x27;-&#x27;</span>, inplace=<span class="literal">True</span>)  <span class="comment"># 所有分母为0或其他情况导致的inf和nan都转换为&#x27;-&#x27;</span></span><br><span class="line">        chart = echarts_stackbar(df=df_abs,</span><br><span class="line">                                 df_gr=df_gr</span><br><span class="line">                                 )  <span class="comment"># 调用stackbar方法生成Pyecharts图表对象</span></span><br><span class="line">        <span class="keyword">return</span> chart.dump_options()  <span class="comment"># 用json格式返回Pyecharts图表对象的全局设置</span></span><br><span class="line">    <span class="keyword">elif</span> chart_type == <span class="string">&#x27;bubble_performance&#x27;</span>:</span><br><span class="line">        df_abs = df.iloc[-<span class="number">1</span>,:]  <span class="comment"># 获取最新时间粒度的绝对值</span></span><br><span class="line">        df_share = df.transform(<span class="keyword">lambda</span> x: x / x.<span class="built_in">sum</span>(), axis=<span class="number">1</span>).iloc[-<span class="number">1</span>,:] <span class="comment"># 获取份额</span></span><br><span class="line">        df_diff = df.diff(periods=<span class="number">4</span>).iloc[-<span class="number">1</span>,:]  <span class="comment"># 获取同比净增长</span></span><br><span class="line"></span><br><span class="line">        chart = mpl_bubble(x=df_abs,  <span class="comment"># x轴数据</span></span><br><span class="line">                           y=df_diff,  <span class="comment"># y轴数据</span></span><br><span class="line">                           z=df_share * <span class="number">50000</span>,  <span class="comment"># 气泡大小数据</span></span><br><span class="line">                           labels=df.columns.<span class="built_in">str</span>.split(<span class="string">&#x27;|&#x27;</span>).<span class="built_in">str</span>[<span class="number">0</span>],  <span class="comment"># 标签数据</span></span><br><span class="line">                           title=<span class="string">&#x27;&#x27;</span>,  <span class="comment"># 图表标题</span></span><br><span class="line">                           x_title=label,  <span class="comment"># x轴标题</span></span><br><span class="line">                           y_title=label + <span class="string">&#x27;净增长&#x27;</span>,  <span class="comment"># y轴标题</span></span><br><span class="line">                           x_fmt=<span class="string">&#x27;&#123;:,.0f&#125;&#x27;</span>,  <span class="comment"># x轴格式</span></span><br><span class="line">                           y_fmt=<span class="string">&#x27;&#123;:,.0f&#125;&#x27;</span>,  <span class="comment"># y轴格式</span></span><br><span class="line">                           y_avg_line=<span class="literal">True</span>,  <span class="comment"># 添加y轴分隔线</span></span><br><span class="line">                           y_avg_value=<span class="number">0</span>,  <span class="comment"># y轴分隔线为y=0</span></span><br><span class="line">                           label_limit=<span class="number">30</span>  <span class="comment"># 只显示前30个项目的标签</span></span><br><span class="line">                           )</span><br><span class="line">        <span class="keyword">return</span> chart</span><br></pre></td></tr></table></figure><p>再度扩展query方法，再增加一个context：</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">def</span> <span class="title function_">query</span>(<span class="params">request</span>):</span><br><span class="line">    ...</span><br><span class="line"></span><br><span class="line">    <span class="comment"># Matplotlib静态图表</span></span><br><span class="line">    bubble_performance = prepare_chart(pivoted, <span class="string">&#x27;bubble_performance&#x27;</span>, form_dict)</span><br><span class="line">    context = &#123;</span><br><span class="line">        ...</span><br><span class="line">        <span class="string">&#x27;bubble_performance&#x27;</span>: bubble_performance</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="keyword">return</span> HttpResponse(json.dumps(context, ensure_ascii=<span class="literal">False</span>), content_type=<span class="string">&quot;application/json charset=utf-8&quot;</span>) <span class="comment"># 返回结果必须是json格式</span></span><br></pre></td></tr></table></figure><p>再看看此时的query结果，下方多了大段字符串为静态图片的base64编码：<br><img src="/images/python-djangosqlpand/v2-a2976c2a2d526959da8f4025f3f11503_1440w.webp"></p><p>之后的思路都和上一章很像，只是实现方法不一样。在display.html模板中，添加一个展示气泡图的tab，重点是留一个空的<img>标签：</p><figure class="highlight html"><table><tr><td class="code"><pre><span class="line"><span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;ui pointing secondary menu&quot;</span>&gt;</span></span><br><span class="line">    ...</span><br><span class="line">    <span class="tag">&lt;<span class="name">a</span> <span class="attr">class</span>=<span class="string">&quot;item&quot;</span> <span class="attr">data-tab</span>=<span class="string">&quot;bubble_performance&quot;</span>&gt;</span><span class="tag">&lt;<span class="name">i</span> <span class="attr">class</span>=<span class="string">&quot;braille icon&quot;</span>&gt;</span><span class="tag">&lt;/<span class="name">i</span>&gt;</span>规模 vs. 增长<span class="tag">&lt;/<span class="name">a</span>&gt;</span></span><br><span class="line"><span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">...</span><br><span class="line"><span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;ui tab segment&quot;</span> <span class="attr">data-tab</span>=<span class="string">&quot;bubble_performance&quot;</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">h3</span> <span class="attr">class</span>=<span class="string">&quot;ui header&quot;</span>&gt;</span></span><br><span class="line">        <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;content&quot;</span>&gt;</span></span><br><span class="line">            规模 versus 增长</span><br><span class="line">            <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;sub header&quot;</span>&gt;</span>带线性拟合的气泡图<span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">        <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;/<span class="name">h3</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;ui divider&quot;</span>&gt;</span><span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;ui container&quot;</span>&gt;</span></span><br><span class="line">        <span class="tag">&lt;<span class="name">img</span> <span class="attr">id</span>=<span class="string">&quot;bubble_performance&quot;</span> <span class="attr">style</span>=<span class="string">&quot;width: 100%&quot;</span> <span class="attr">alt</span>=<span class="string">&quot;&quot;</span> /&gt;</span></span><br><span class="line">    <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line"><span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br></pre></td></tr></table></figure><p>注意上方<img>标签中的style&#x3D;”width: 100%”，没有这句图片将展示原始尺寸。</p><p>最后再次追加filter.html中AJAX部分的回调函数，用jQuery的.attr语法修改之前预留<img>中的src参数：</p><figure class="highlight js"><table><tr><td class="code"><pre><span class="line">&lt;script type=<span class="string">&quot;text/javascript&quot;</span>&gt;</span><br><span class="line">    $(<span class="string">&quot;#AJAX_get&quot;</span>).<span class="title function_">click</span>(<span class="keyword">function</span> (<span class="params">event</span>) &#123;</span><br><span class="line">        ...</span><br><span class="line"></span><br><span class="line">        $.<span class="title function_">ajax</span>(&#123;</span><br><span class="line">            ...</span><br><span class="line">            <span class="attr">success</span>: <span class="keyword">function</span> (<span class="params">ret</span>) &#123;     <span class="comment">//成功执行</span></span><br><span class="line">                ...</span><br><span class="line">                <span class="comment">// 展示Matplotlib气泡图</span></span><br><span class="line">                $(<span class="string">&quot;#bubble_performance&quot;</span>).<span class="title function_">attr</span>(<span class="string">&#x27;src&#x27;</span>, ret[<span class="string">&#x27;bubble_performance&#x27;</span>]);</span><br><span class="line">            &#125;,</span><br><span class="line">            ...</span><br><span class="line">        &#125;);</span><br><span class="line">    &#125;)</span><br><span class="line">&lt;/script&gt;</span><br></pre></td></tr></table></figure><p>大功告成，我们现在可以动态生成任意一个领域的复杂统计图表了。看看我们一直用来测试的高血压ARB市场的产品：</p>]]>
    </content>
    <id>https://blog.malu.tech/python-django-sql-pandas-pyecharts-data-analysis-platform-10/</id>
    <link href="https://blog.malu.tech/python-django-sql-pandas-pyecharts-data-analysis-platform-10/"/>
    <published>2024-03-03T16:00:00.000Z</published>
    <summary>Python Django+SQL+Pandas+Pyecharts自建在线数据分析平台系列文章第十篇：静态图表的展示</summary>
    <title>Python Django+SQL+Pandas+Pyecharts自建在线数据分析平台（十）</title>
    <updated>2024-03-03T16:00:00.000Z</updated>
  </entry>
  <entry>
    <author>
      <name>Road</name>
    </author>
    <category term="Python" scheme="https://blog.malu.tech/categories/Python/"/>
    <category term="Django" scheme="https://blog.malu.tech/categories/Python/Django/"/>
    <category term="数据分析" scheme="https://blog.malu.tech/categories/Python/Django/%E6%95%B0%E6%8D%AE%E5%88%86%E6%9E%90/"/>
    <category term="Python" scheme="https://blog.malu.tech/tags/Python/"/>
    <category term="Django" scheme="https://blog.malu.tech/tags/Django/"/>
    <category term="SQL" scheme="https://blog.malu.tech/tags/SQL/"/>
    <category term="Pandas" scheme="https://blog.malu.tech/tags/Pandas/"/>
    <category term="Pyecharts" scheme="https://blog.malu.tech/tags/Pyecharts/"/>
    <category term="数据分析平台" scheme="https://blog.malu.tech/tags/%E6%95%B0%E6%8D%AE%E5%88%86%E6%9E%90%E5%B9%B3%E5%8F%B0/"/>
    <content>
      <![CDATA[<blockquote><p><strong>文章来源</strong>：<a href="https://zhuanlan.zhihu.com/p/142490087">Python Django+SQL+Pandas+Pyecharts自建在线数据分析平台（一）</a><br><strong>作者</strong>：<a href="https://www.zhihu.com/people/chen-cheng-76-40">ccpic</a><br><strong>感谢</strong>：感谢作者 ccpic 分享的优质内容，本网页主要用于学习知识的存档备份，欢迎点击原网页支持作者。</p></blockquote><p>（一）需求分析&amp;技术实现</p><p>（二）初步搭建Django环境</p><p>（三）页面布局&amp;Django模板</p><p>（四）SQL+Pandas初步处理数据</p><p>（五）前端表单交互</p><p>（六）Ajax异步传参与加载</p><p>（七）前端数据格式的处理</p><p>（八）DataTables接管前端表格</p><p>（十）静态图表的展示</p><p>（十一）“导出数据至Excel”功能</p><p>（十二）添加和配置缓存</p><p>（十三）用户登录系统</p><p>（十四）部署Django至生产环境</p><p>Echarts是一个由百度开源的纯JS的数据可视化库，而Pyecharts可以理解为一个Echarts的Python封装版本。我觉得Pyecharts的存在是很有必要的，因为Python作为近年来数据领域最常用的语言，数据处理后正好利用其无缝衔接到数据可视化这个后续步骤。</p><p>另外让人感到亲切的一点是，Echarts和Pyecharts都是国人开发和开源的。</p><p>在Pyecharts的官方文档中，就有和Django整合的例子：</p><p>虽然此例子中用的2种实现方式——渲染整体模板和rest API类视图，都不太适合我们这个项目。但是我们可以看出Pyecharts和Django整合的基本套路：</p><ol><li><strong>前端html部分准备一个空白的DOM</strong></li><li><strong>JS部分用echarts初始化命令渲染这个元素</strong></li><li><strong>后端准备数据</strong></li><li><strong>使用数据用Pyecharts生成图表对象</strong></li><li><strong>利用Pyechart全局API dump_options()生成图表对象json格式的全局options</strong></li><li><strong>前端JS部分用AJAX通信获取后端json</strong></li><li><strong>AJAX调用json成功后使用.setOption方法刷新图表元素呈现可视化结果</strong></li></ol><p>明白了以上这些步骤，可以在我们的项目里有条不紊地实现任何Pyecharts交互图表。本章的目标为创建一个条形图和折线图的组合图，呈现我们query方法返回的整体市场趋势及同比增长率。以下每个具体步骤都与上方对应：</p><p>1、在display.html模板内加入下方代码创造一个布局片段包含一个id为bar_total_trend的空白DOM：</p><figure class="highlight html"><table><tr><td class="code"><pre><span class="line"><span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;ui tab segment active&quot;</span> <span class="attr">data-tab</span>=<span class="string">&quot;total&quot;</span>&gt;</span></span><br><span class="line">    ...</span><br><span class="line">    <span class="tag">&lt;<span class="name">h3</span> <span class="attr">class</span>=<span class="string">&quot;ui header&quot;</span>&gt;</span></span><br><span class="line">        <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;content&quot;</span>&gt;</span></span><br><span class="line">            定义市场总量趋势</span><br><span class="line">            <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;sub header&quot;</span>&gt;</span>柱状折线复合图<span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">        <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;/<span class="name">h3</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;ui divider&quot;</span>&gt;</span><span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;ui container&quot;</span>&gt;</span></span><br><span class="line">        <span class="tag">&lt;<span class="name">div</span> <span class="attr">id</span>=<span class="string">&quot;bar_total_trend&quot;</span> <span class="attr">style</span>=<span class="string">&quot;width:1000px; height:600px;&quot;</span>&gt;</span><span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line"><span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br></pre></td></tr></table></figure><p>2、在filter.html的JS部分（注意因为Django模板的继承特性，他们是互通的，也就是filter.html的JS代码能控制display.html的DOM），初始化上面的DOM元素为echarts对象：</p><figure class="highlight js"><table><tr><td class="code"><pre><span class="line">&lt;script type=<span class="string">&quot;text/javascript&quot;</span>&gt;</span><br><span class="line">    $(<span class="string">&quot;#AJAX_get&quot;</span>).<span class="title function_">click</span>(<span class="keyword">function</span> (<span class="params"></span>) &#123;</span><br><span class="line">        ...</span><br><span class="line">        <span class="comment">// Pyecharts图表初始化</span></span><br><span class="line">        <span class="keyword">var</span> chart = echarts.<span class="title function_">init</span>(<span class="variable language_">document</span>.<span class="title function_">getElementById</span>(<span class="string">&#x27;bar_total_trend&#x27;</span>), <span class="string">&#x27;white&#x27;</span>, &#123;<span class="attr">renderer</span>: <span class="string">&#x27;canvas&#x27;</span>&#125;);</span><br><span class="line">        chart.<span class="title function_">showLoading</span>(&#123;</span><br><span class="line">          text : <span class="string">&#x27;正在加载数据&#x27;</span></span><br><span class="line">        &#125;);  <span class="comment">//增加加载提示</span></span><br><span class="line">        ...</span><br><span class="line">    &#125;)</span><br><span class="line">&lt;/script&gt;</span><br></pre></td></tr></table></figure><p>3&amp;4&amp;5、后端views.py编写一个prepare_chart方法，可以根据字符串图表类型准备数据以及Pyecharts的图表对象：</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">from</span> .charts <span class="keyword">import</span> *</span><br><span class="line"></span><br><span class="line">D_TRANS = &#123;</span><br><span class="line">            <span class="string">&#x27;MAT&#x27;</span>: <span class="string">&#x27;滚动年&#x27;</span>,</span><br><span class="line">            <span class="string">&#x27;QTR&#x27;</span>: <span class="string">&#x27;季度&#x27;</span>,</span><br><span class="line">            <span class="string">&#x27;Value&#x27;</span>: <span class="string">&#x27;金额&#x27;</span>,</span><br><span class="line">            <span class="string">&#x27;Volume&#x27;</span>: <span class="string">&#x27;盒数&#x27;</span>,</span><br><span class="line">            <span class="string">&#x27;Volume (Counting Unit)&#x27;</span>: <span class="string">&#x27;最小制剂单位数&#x27;</span>,</span><br><span class="line">            <span class="string">&#x27;滚动年&#x27;</span>: <span class="string">&#x27;MAT&#x27;</span>,</span><br><span class="line">            <span class="string">&#x27;季度&#x27;</span>: <span class="string">&#x27;QTR&#x27;</span>,</span><br><span class="line">            <span class="string">&#x27;金额&#x27;</span>: <span class="string">&#x27;Value&#x27;</span>,</span><br><span class="line">            <span class="string">&#x27;盒数&#x27;</span>: <span class="string">&#x27;Volume&#x27;</span>,</span><br><span class="line">            <span class="string">&#x27;最小制剂单位数&#x27;</span>: <span class="string">&#x27;Volume (Counting Unit)&#x27;</span></span><br><span class="line">           &#125;</span><br><span class="line"></span><br><span class="line"><span class="keyword">def</span> <span class="title function_">prepare_chart</span>(<span class="params">df,  <span class="comment"># 输入经过pivoted方法透视过的df，不是原始df</span></span></span><br><span class="line"><span class="params">                  chart_type,  <span class="comment"># 图表类型字符串，人为设置，根据图表类型不同做不同的Pandas数据处理，及生成不同的Pyechart对象</span></span></span><br><span class="line"><span class="params">                  form_dict,  <span class="comment"># 前端表单字典，用来获得一些变量作为图表的标签如单位</span></span></span><br><span class="line"><span class="params">                  </span>):</span><br><span class="line">    label = D_TRANS[form_dict[<span class="string">&#x27;PERIOD_select&#x27;</span>][<span class="number">0</span>]] + D_TRANS[form_dict[<span class="string">&#x27;UNIT_select&#x27;</span>][<span class="number">0</span>]]</span><br><span class="line"></span><br><span class="line">    <span class="keyword">if</span> chart_type == <span class="string">&#x27;bar_total_trend&#x27;</span>:</span><br><span class="line">        df_abs = df.<span class="built_in">sum</span>(axis=<span class="number">1</span>)  <span class="comment"># Pandas列汇总，返回一个N行1列的series，每行是一个date的市场综合</span></span><br><span class="line">        df_abs.index = df_abs.index.strftime(<span class="string">&quot;%Y-%m&quot;</span>)  <span class="comment"># 行索引日期数据变成2020-06的形式</span></span><br><span class="line">        df_abs = df_abs.to_frame()  <span class="comment"># series转换成df</span></span><br><span class="line">        df_abs.columns = [label]  <span class="comment"># 用一些设置变量为系列命名，准备作为图表标签</span></span><br><span class="line">        df_gr = df_abs.pct_change(periods=<span class="number">4</span>)  <span class="comment"># 获取同比增长率</span></span><br><span class="line">        df_gr.dropna(how=<span class="string">&#x27;all&#x27;</span>, inplace=<span class="literal">True</span>)  <span class="comment"># 删除没有同比增长率的行，也就是时间序列数据的最前面几行，他们没有同比</span></span><br><span class="line">        df_gr.replace([np.inf, -np.inf, np.nan], <span class="string">&#x27;-&#x27;</span>, inplace=<span class="literal">True</span>)  <span class="comment"># 所有分母为0或其他情况导致的inf和nan都转换为&#x27;-&#x27;</span></span><br><span class="line">        chart = echarts_stackbar(df=df_abs,</span><br><span class="line">                                 df_gr=df_gr</span><br><span class="line">                                 )  <span class="comment"># 调用stackbar方法生成Pyecharts图表对象</span></span><br><span class="line">        <span class="keyword">return</span> chart.dump_options()  <span class="comment"># 用json格式返回Pyecharts图表对象的全局设置</span></span><br><span class="line">    <span class="keyword">else</span>:</span><br><span class="line">        <span class="keyword">return</span> <span class="literal">None</span></span><br></pre></td></tr></table></figure><p>这部分最后的.dump_options()是重点，返回的不是图表对象，而是json格式的图表全局设置。</p><p>因为views.py已经有点臃肿了，我们在相同目录新建一个charts.py，把生成Pyecharts图表对象的方法都写到这里。本例里只写了个柱状图和线图的组合：</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">from</span> pyecharts.charts <span class="keyword">import</span> Line, Bar</span><br><span class="line"><span class="keyword">from</span> pyecharts <span class="keyword">import</span> options <span class="keyword">as</span> opts</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="keyword">def</span> <span class="title function_">echarts_stackbar</span>(<span class="params">df,  <span class="comment"># 传入数据df，应该是一个行索引为date的时间序列面板数据</span></span></span><br><span class="line"><span class="params">             df_gr=<span class="literal">None</span>,  <span class="comment"># 传入同比增长率df，可以没有</span></span></span><br><span class="line"><span class="params">             datatype=<span class="string">&#x27;ABS&#x27;</span>  <span class="comment"># 主Y轴形式是绝对值，增长率还是份额，用来确定一些标签格式，默认为绝对值</span></span></span><br><span class="line"><span class="params">             </span>) -&gt; Bar:</span><br><span class="line"></span><br><span class="line">    axislabel_format = <span class="string">&#x27;&#123;value&#125;&#x27;</span>  <span class="comment"># 主Y轴默认格式</span></span><br><span class="line">    <span class="built_in">max</span> = df[df&gt;<span class="number">0</span>].<span class="built_in">sum</span>(axis=<span class="number">1</span>).<span class="built_in">max</span>()  <span class="comment"># 主Y轴默认最大值</span></span><br><span class="line">    <span class="built_in">min</span> = df[df&lt;=<span class="number">0</span>].<span class="built_in">sum</span>(axis=<span class="number">1</span>).<span class="built_in">min</span>()  <span class="comment"># 主Y轴默认最小值</span></span><br><span class="line">    <span class="keyword">if</span> datatype <span class="keyword">in</span> [<span class="string">&#x27;SHARE&#x27;</span>, <span class="string">&#x27;GR&#x27;</span>]:  <span class="comment"># 如果主数据不是绝对值形式而是份额或增长率如何处理</span></span><br><span class="line">        df = df.multiply(<span class="number">100</span>).<span class="built_in">round</span>(<span class="number">2</span>)</span><br><span class="line">        axislabel_format = <span class="string">&#x27;&#123;value&#125;%&#x27;</span></span><br><span class="line">        <span class="built_in">max</span> = <span class="number">100</span></span><br><span class="line">        <span class="built_in">min</span> = <span class="number">0</span></span><br><span class="line">    <span class="keyword">if</span> df_gr <span class="keyword">is</span> <span class="keyword">not</span> <span class="literal">None</span>:</span><br><span class="line">        df_gr = df_gr.multiply(<span class="number">100</span>).<span class="built_in">round</span>(<span class="number">2</span>) <span class="comment"># 如果有同比增长率，原始数*100呈现</span></span><br><span class="line"></span><br><span class="line">    <span class="keyword">if</span> df.empty <span class="keyword">is</span> <span class="literal">False</span>:</span><br><span class="line">        stackbar = (</span><br><span class="line">            Bar()</span><br><span class="line">            .add_xaxis(df.index.tolist())</span><br><span class="line">        )</span><br><span class="line">        <span class="keyword">for</span> i, item <span class="keyword">in</span> <span class="built_in">enumerate</span>(df.columns): <span class="comment"># 预留的枚举，这个方法以后可以根据输入对象不同从单一柱状图变成堆积柱状图</span></span><br><span class="line">            stackbar.add_yaxis(item,</span><br><span class="line">                          df[item].values.tolist(),</span><br><span class="line">                          stack=<span class="string">&#x27;总量&#x27;</span>,</span><br><span class="line">                          label_opts=opts.LabelOpts(is_show=<span class="literal">False</span>),</span><br><span class="line">                          z_level=<span class="number">1</span>,  <span class="comment"># 指定渲染图层，低版本pyecharts可能因为没有该参数报错</span></span><br><span class="line">                          )</span><br><span class="line">        <span class="keyword">if</span> df_gr <span class="keyword">is</span> <span class="keyword">not</span> <span class="literal">None</span>:  <span class="comment"># 如果有同比增长率数据则加入次Y轴</span></span><br><span class="line">            stackbar.extend_axis(</span><br><span class="line">                yaxis=opts.AxisOpts(</span><br><span class="line">                    name=<span class="string">&quot;同比增长率&quot;</span>,</span><br><span class="line">                    type_=<span class="string">&quot;value&quot;</span>,</span><br><span class="line">                    axislabel_opts=opts.LabelOpts(formatter=<span class="string">&quot;&#123;value&#125;%&quot;</span>),</span><br><span class="line">                )</span><br><span class="line">            )</span><br><span class="line">        stackbar.set_global_opts(</span><br><span class="line">            legend_opts=opts.LegendOpts(pos_top=<span class="string">&#x27;5%&#x27;</span>, pos_left=<span class="string">&#x27;10%&#x27;</span>, pos_right=<span class="string">&#x27;60%&#x27;</span>),</span><br><span class="line">            toolbox_opts=opts.ToolboxOpts(is_show=<span class="literal">True</span>),</span><br><span class="line">            tooltip_opts=opts.TooltipOpts(trigger=<span class="string">&#x27;axis&#x27;</span>,</span><br><span class="line">                                          axis_pointer_type=<span class="string">&#x27;cross&#x27;</span>,</span><br><span class="line">                                          ),</span><br><span class="line">            xaxis_opts=opts.AxisOpts(type_=<span class="string">&#x27;category&#x27;</span>,</span><br><span class="line">                                     boundary_gap=<span class="literal">True</span>,</span><br><span class="line">                                     axislabel_opts=opts.LabelOpts(rotate=<span class="number">90</span>), <span class="comment"># x轴标签方向rotate有时能解决拥挤显示不全的问题</span></span><br><span class="line">                                     splitline_opts=opts.SplitLineOpts(is_show=<span class="literal">False</span>,</span><br><span class="line">                                                                       linestyle_opts=opts.LineStyleOpts(</span><br><span class="line">                                                                           type_=<span class="string">&#x27;dotted&#x27;</span>,</span><br><span class="line">                                                                           opacity=<span class="number">0.5</span>,</span><br><span class="line">                                                                       )</span><br><span class="line">                                                                       )</span><br><span class="line">                                     ),</span><br><span class="line">            yaxis_opts=opts.AxisOpts(max_=<span class="built_in">max</span>,</span><br><span class="line">                                     min_=<span class="built_in">min</span>,</span><br><span class="line">                                     type_=<span class="string">&quot;value&quot;</span>,</span><br><span class="line">                                     axislabel_opts=opts.LabelOpts(formatter=axislabel_format),</span><br><span class="line">                                     <span class="comment"># axistick_opts=opts.AxisTickOpts(is_show=True),</span></span><br><span class="line">                                     splitline_opts=opts.SplitLineOpts(is_show=<span class="literal">True</span>,</span><br><span class="line">                                                                       linestyle_opts=opts.LineStyleOpts(</span><br><span class="line">                                                                           type_=<span class="string">&#x27;dotted&#x27;</span>,</span><br><span class="line">                                                                           opacity=<span class="number">0.5</span>,</span><br><span class="line">                                                                       )</span><br><span class="line">                                                                       )</span><br><span class="line">                                     ),</span><br><span class="line">        )</span><br><span class="line">        <span class="keyword">if</span> df_gr <span class="keyword">is</span> <span class="keyword">not</span> <span class="literal">None</span>:</span><br><span class="line">            line = (</span><br><span class="line">                Line()</span><br><span class="line">                    .add_xaxis(xaxis_data=df_gr.index.tolist())</span><br><span class="line">                    .add_yaxis(</span><br><span class="line">                    series_name=<span class="string">&quot;同比增长率&quot;</span>,</span><br><span class="line">                    yaxis_index=<span class="number">1</span>,</span><br><span class="line">                    y_axis=df_gr.values.tolist(),</span><br><span class="line">                    label_opts=opts.LabelOpts(is_show=<span class="literal">False</span>),</span><br><span class="line">                    linestyle_opts=opts.LineStyleOpts(width=<span class="number">3</span>),</span><br><span class="line">                    symbol_size=<span class="number">8</span>,</span><br><span class="line">                    itemstyle_opts=opts.ItemStyleOpts(border_width=<span class="number">1</span>, border_color=<span class="string">&#x27;&#x27;</span>, border_color0=<span class="string">&#x27;white&#x27;</span>),</span><br><span class="line">                    z_level=<span class="number">2</span>  <span class="comment"># 渲染图层大于柱状图，保证线图在上方，低版本pyecharts可能因为没有该参数报错</span></span><br><span class="line">               )</span><br><span class="line">            )</span><br><span class="line">    <span class="keyword">else</span>:</span><br><span class="line">        stackbar = (Bar())</span><br><span class="line"></span><br><span class="line">    <span class="keyword">if</span> df_gr <span class="keyword">is</span> <span class="keyword">not</span> <span class="literal">None</span>:</span><br><span class="line">        <span class="keyword">return</span> stackbar.overlap(line) <span class="comment"># 如果有次坐标轴最后要用overlap方法组合</span></span><br><span class="line">    <span class="keyword">else</span>:</span><br><span class="line">        <span class="keyword">return</span> stackbar</span><br></pre></td></tr></table></figure><p>6&amp;7、在views.py的query方法里的context增加之前步骤生成的图表options，切记一定要整合在query方法内，如果另起炉灶，读取数据的步骤要重复一遍，等于性能减半：</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">def</span> <span class="title function_">query</span>(<span class="params">request</span>):</span><br><span class="line">    ...</span><br><span class="line"></span><br><span class="line">    <span class="comment"># Pyecharts交互图表</span></span><br><span class="line">    bar_total_trend = json.loads(prepare_chart(pivoted, <span class="string">&#x27;bar_total_trend&#x27;</span>, form_dict))</span><br><span class="line"></span><br><span class="line">    context = &#123;</span><br><span class="line">        ...</span><br><span class="line">        <span class="string">&#x27;bar_total_trend&#x27;</span>: bar_total_trend</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="keyword">return</span> HttpResponse(json.dumps(context, ensure_ascii=<span class="literal">False</span>), content_type=<span class="string">&quot;application/json charset=utf-8&quot;</span>) <span class="comment"># 返回结果必须是json格式</span></span><br></pre></td></tr></table></figure><p>可以看看这时的query方法返回了什么，可以看到下方大段的echarts options：<br><img src="/images/python-djangosqlpand/v2-801357a13f312f8e9991e534f234bd3a_1440w.webp"></p><p>回到filter.html的JS AJAX部分，在回调函数部分加入以下代码：</p><figure class="highlight js"><table><tr><td class="code"><pre><span class="line">&lt;script type=<span class="string">&quot;text/javascript&quot;</span>&gt;</span><br><span class="line">    $(<span class="string">&quot;#AJAX_get&quot;</span>).<span class="title function_">click</span>(<span class="keyword">function</span> (<span class="params">event</span>) &#123;</span><br><span class="line">        ...</span><br><span class="line">        $.<span class="title function_">ajax</span>(&#123;</span><br><span class="line">            ...</span><br><span class="line">            <span class="attr">success</span>: <span class="keyword">function</span> (<span class="params">ret</span>) &#123;     <span class="comment">//成功执行</span></span><br><span class="line">                <span class="comment">// 展示Pyecharts整体市场柱状组合图</span></span><br><span class="line">                chart.<span class="title function_">clear</span>();</span><br><span class="line">                chart.<span class="title function_">setOption</span>(ret[<span class="string">&#x27;bar_total_trend&#x27;</span>]);</span><br><span class="line">                chart.<span class="title function_">hideLoading</span>()</span><br><span class="line">            &#125;,</span><br><span class="line">            ...</span><br><span class="line">        &#125;);</span><br><span class="line">    &#125;)</span><br><span class="line">&lt;/script&gt;</span><br></pre></td></tr></table></figure><p>我们的第一个交互图表就诞生了：<br><img src="/images/python-djangosqlpand/v2-e195b7d01fe34459bb545c5354b78ded_1440w.webp"></p>]]>
    </content>
    <id>https://blog.malu.tech/python-django-sql-pandas-pyecharts-data-analysis-platform-9/</id>
    <link href="https://blog.malu.tech/python-django-sql-pandas-pyecharts-data-analysis-platform-9/"/>
    <published>2024-02-25T16:00:00.000Z</published>
    <summary>Python Django+SQL+Pandas+Pyecharts自建在线数据分析平台系列文章第九篇：Pyecharts实现交互图表</summary>
    <title>Python Django+SQL+Pandas+Pyecharts自建在线数据分析平台（九）</title>
    <updated>2024-02-25T16:00:00.000Z</updated>
  </entry>
  <entry>
    <author>
      <name>Road</name>
    </author>
    <category term="Python" scheme="https://blog.malu.tech/categories/Python/"/>
    <category term="Django" scheme="https://blog.malu.tech/categories/Python/Django/"/>
    <category term="数据分析" scheme="https://blog.malu.tech/categories/Python/Django/%E6%95%B0%E6%8D%AE%E5%88%86%E6%9E%90/"/>
    <category term="Python" scheme="https://blog.malu.tech/tags/Python/"/>
    <category term="Django" scheme="https://blog.malu.tech/tags/Django/"/>
    <category term="SQL" scheme="https://blog.malu.tech/tags/SQL/"/>
    <category term="Pandas" scheme="https://blog.malu.tech/tags/Pandas/"/>
    <category term="Pyecharts" scheme="https://blog.malu.tech/tags/Pyecharts/"/>
    <category term="数据分析平台" scheme="https://blog.malu.tech/tags/%E6%95%B0%E6%8D%AE%E5%88%86%E6%9E%90%E5%B9%B3%E5%8F%B0/"/>
    <content>
      <![CDATA[<blockquote><p><strong>文章来源</strong>：<a href="https://zhuanlan.zhihu.com/p/142490087">Python Django+SQL+Pandas+Pyecharts自建在线数据分析平台（一）</a><br><strong>作者</strong>：<a href="https://www.zhihu.com/people/chen-cheng-76-40">ccpic</a><br><strong>感谢</strong>：感谢作者 ccpic 分享的优质内容，本网页主要用于学习知识的存档备份，欢迎点击原网页支持作者。</p></blockquote><p>（一）需求分析&amp;技术实现</p><p>（二）初步搭建Django环境</p><p>（三）页面布局&amp;Django模板</p><p>（四）SQL+Pandas初步处理数据</p><p>（五）前端表单交互</p><p>（六）Ajax异步传参与加载</p><p>（七）前端数据格式的处理</p><h2 id="（八）DataTables接管前端表格"><a href="#（八）DataTables接管前端表格" class="headerlink" title="（八）DataTables接管前端表格"></a>（八）DataTables接管前端表格</h2><p>（九）Pyecharts实现交互图表</p><p>（十）静态图表的展示</p><p>（十一）“导出数据至Excel”功能</p><p>（十二）添加和配置缓存</p><p>（十三）用户登录系统</p><p>（十四）部署Django至生产环境</p><p>我一直认为表格是数据分析可视化的最重要形式，是整个项目最终成果的重中之重，因此放到交互图表之前来说。分析人员应该把制造酷炫的dashboard的激情分一点到表格上来，而呈现一个好的表格实际是一个非常综合且细节的用户体验管理，既要呈现美观醒目的结果，又要包含丰富实用的功能。</p><p>一个优秀的表格解决方案绝对不是个人就能搞定的小工程。因此，本章尝试用jQuery的<u><a href="https://link.zhihu.com/?target=http%3A//www.datatables.net/" target="_blank" rel="nofollow noreferrer">DataTables</a></u>插件接管前端表格，DataTables可能不是最后呈现效果最好的方案，甚至有些过时。但是胜在高度灵活，功能完善，社区活跃，基本上你想到的都能实现。本章的内容也会主要聚焦在一些细节的雕琢上，争取用最简单的方式获取尽可能佳的用户体验。</p><p>DataTables恰巧有我们Semantic UI的主题，那我们就使用它使整体风格更统一。在DataTables官网下载时记得要勾选，下载下来的css才会是Semantic UI主题的：<br><img src="/images/python-djangosqlpand/v2-f604e4d0feda9d827a07cba9d5e8c767_1440w.webp"></p><p>在第3章的base.html模板里，下面的引用部分都是和DataTables相关的。我们多下了一个百分比条形图插件percentageBars，接下来会用到：</p><figure class="highlight js"><table><tr><td class="code"><pre><span class="line">&lt;!-- <span class="title class_">DataTables</span> <span class="title class_">Semantic</span> <span class="variable constant_">UI</span>主题的<span class="variable constant_">CSS</span> --&gt;</span><br><span class="line"><span class="language-xml"><span class="tag">&lt;<span class="name">link</span> <span class="attr">rel</span>=<span class="string">&quot;stylesheet&quot;</span> <span class="attr">href</span>=<span class="string">&quot;&#123;<span class="symbol">&amp;#37;</span> static &#x27;datatables/dataTables.semanticui.min.css&#x27; <span class="symbol">&amp;#37;</span>&#125;&quot;</span>&gt;</span></span></span><br><span class="line"><span class="language-xml"><span class="comment">&lt;!-- DataTables 主JS --&gt;</span></span></span><br><span class="line"><span class="language-xml"><span class="tag">&lt;<span class="name">script</span> <span class="attr">src</span>=<span class="string">&quot;&#123;<span class="symbol">&amp;#37;</span> static &#x27;datatables/jquery.dataTables.min.js&#x27; <span class="symbol">&amp;#37;</span>&#125;&quot;</span>&gt;</span><span class="tag">&lt;/<span class="name">script</span>&gt;</span></span></span><br><span class="line"><span class="language-xml"><span class="comment">&lt;!-- DataTables Semantic UI主题的JS --&gt;</span></span></span><br><span class="line"><span class="language-xml"><span class="tag">&lt;<span class="name">script</span> <span class="attr">src</span>=<span class="string">&quot;&#123;<span class="symbol">&amp;#37;</span> static &#x27;datatables/dataTables.semanticui.min.js&#x27; <span class="symbol">&amp;#37;</span>&#125;&quot;</span>&gt;</span><span class="tag">&lt;/<span class="name">script</span>&gt;</span></span></span><br><span class="line"><span class="language-xml"><span class="comment">&lt;!-- DataTables 百分数条形图插件 --&gt;</span></span></span><br><span class="line"><span class="language-xml"><span class="tag">&lt;<span class="name">script</span> <span class="attr">src</span>=<span class="string">&quot;&#123;<span class="symbol">&amp;#37;</span> static &#x27;datatables/percentageBars.js&#x27; <span class="symbol">&amp;#37;</span>&#125;&quot;</span>&gt;</span><span class="tag">&lt;/<span class="name">script</span>&gt;</span></span></span><br></pre></td></tr></table></figure><p>DataTables初始化的方式非常灵活多变，主要差别在于：</p><ul><li><strong>数据源是DOM，AJAX json, js还是server-side processing</strong></li><li><strong>分页是在前端还是服务器端</strong></li><li><strong>初始化的设置是全部js执行还是html里指定列title的方式</strong></li></ul><p>我的建议当然还是根据应用场景因地制宜，本次项目我们之前的操作都是通过Pandas的.to_html()方法传一整个DOM表格（如果用drf做api则DataTabels初始化方法会完全不同）。一般分析的结果也不会有很多行不需要服务器端分页（但在表格条目特别多的场景服务器端分页然后异步加载则异常重要）。</p><p>总之，我们最后选择使用数据源为DOM，前端分页，JS设置的方法。</p><p>首先，在前两章的table.to_html()方法里多声明2个参数，给AJAX传的DOM表格指定一个css class和id：</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">def</span> <span class="title function_">query</span>(<span class="params">request</span>):</span><br><span class="line">    ...</span><br><span class="line">    table = table.to_html(formatters=build_formatters_by_col(table),  <span class="comment"># 逐列调整表格内数字格式</span></span><br><span class="line">                          classes=<span class="string">&#x27;ui selectable celled table&#x27;</span>,  <span class="comment"># 指定表格css class为Semantic UI主题</span></span><br><span class="line">                          table_id=<span class="string">&#x27;ptable&#x27;</span>  <span class="comment"># 指定表格id</span></span><br><span class="line">                          )</span><br><span class="line">    ...</span><br></pre></td></tr></table></figure><p>上方代码中的classes&#x3D;’ui selectable celled table’即为Semantic UI的表格样式，可以参考下方页面选择自己喜欢的各种变种：</p><p>在filter.html里的js部分加入一个initTable方法，调用DataTables的默认初始化语句：</p><figure class="highlight js"><table><tr><td class="code"><pre><span class="line">&lt;script&gt;</span><br><span class="line">    <span class="keyword">function</span> <span class="title function_">initTable</span>(<span class="params">table</span>) &#123;</span><br><span class="line">        table.<span class="title class_">DataTable</span>(</span><br><span class="line">            &#123;</span><br><span class="line">            &#125;</span><br><span class="line">        );</span><br><span class="line">    &#125;</span><br><span class="line">&lt;/script&gt;</span><br></pre></td></tr></table></figure><p>再在第六章AJAX回传的success参数里根据后端新指定的id “ptable”指定表格元素并call这个initTable方法就可以了：</p><figure class="highlight js"><table><tr><td class="code"><pre><span class="line">&lt;script type=<span class="string">&quot;text/javascript&quot;</span>&gt;</span><br><span class="line">    $(<span class="string">&quot;#AJAX_get&quot;</span>).<span class="title function_">click</span>(<span class="keyword">function</span> (<span class="params"></span>) &#123;</span><br><span class="line">        ...</span><br><span class="line">        $.<span class="title function_">ajax</span>(&#123;</span><br><span class="line">            ...</span><br><span class="line">            <span class="attr">success</span>: <span class="keyword">function</span> (<span class="params">ret</span>) &#123;     <span class="comment">// 成功执行</span></span><br><span class="line">                ...</span><br><span class="line">                $(<span class="string">&quot;#result_table&quot;</span>).<span class="title function_">html</span>(ret[<span class="string">&#x27;ptable&#x27;</span>]);</span><br><span class="line">                <span class="title function_">initTable</span>($(<span class="string">&quot;#ptable&quot;</span>)) <span class="comment">// 为id为ptable的DOM表格初始化DataTables，即上一行刚刚修改了DOM元素的那个</span></span><br><span class="line">            &#125;,</span><br><span class="line">            ...</span><br><span class="line">        &#125;);</span><br><span class="line">    &#125;)</span><br><span class="line">&lt;/script&gt;</span><br></pre></td></tr></table></figure><p>（注意上方代码块中的连续两行的两个’ptable’是完全不同的，第一行的’ptable’是视图端传来的context字典的键值，而第二行的’ptable’是我们刚刚设定的表的id。）</p><p>此时可以在前端测试下，查询后表格的呈现已经发生变化了，外观改变之余也多了前端分页，搜索筛选，排序等新功能：</p><p>但整体效果和各种细节依然肉眼可见的不完善，我们还有以下诉求：<br><img src="/images/python-djangosqlpand/v2-1b8a03ea0b84123dd02cf2c93bfe27cc_1440w.webp"></p><ul><li><strong>表格整体宽度与父元素一致且保持不变</strong></li><li><strong>默认以第2列（销售额）由高到低排序</strong></li><li><strong>默认前端分页设置为每页呈现25个结果</strong></li><li><strong>所有UI的label本地化为中文显示</strong></li><li><strong>表格内所有负数高亮为红色字体</strong></li><li><strong>第7列（EI）是个描述增速是否跑赢大盘的指标，要求高于100的数值高亮绿色字体，低于100的数值高亮为红色字体</strong></li><li><strong>第4列（份额）用条形图展示，增加醒目感</strong></li><li><strong>增加一个按键一键复制表格，方便粘贴到Excel或其他地方，这是一个常用需求，用鼠标划选麻烦且容易错位。</strong></li></ul><p>前4个问题是DataTables的基本设置就可以搞定的，可以修改initTable方法。DataTables的基本设置已经足以实现很多实用功能了，具体请参考文档：</p><figure class="highlight js"><table><tr><td class="code"><pre><span class="line">&lt;script&gt;</span><br><span class="line">    <span class="keyword">function</span> <span class="title function_">initTable</span>(<span class="params">table</span>) &#123;</span><br><span class="line">        table.<span class="title class_">DataTable</span>(</span><br><span class="line">            &#123;</span><br><span class="line">                <span class="attr">order</span>: [[<span class="number">1</span>, <span class="string">&quot;desc&quot;</span>]], <span class="comment">// 初始以第2列（注意第一列索引为0）由高到低排序</span></span><br><span class="line">                <span class="attr">pageLength</span>: <span class="number">25</span>, <span class="comment">// 前端分页，初始每页显示25条记录</span></span><br><span class="line">                <span class="attr">autoWidth</span>: <span class="literal">false</span>, <span class="comment">// 不自动调整表格宽度</span></span><br><span class="line">                <span class="attr">oLanguage</span>: &#123; <span class="comment">// UI Label本地化</span></span><br><span class="line">                    <span class="string">&quot;sLengthMenu&quot;</span>: <span class="string">&quot;显示 _MENU_ 项结果&quot;</span>,</span><br><span class="line">                    <span class="string">&quot;sProcessing&quot;</span>: <span class="string">&quot;处理中...&quot;</span>,</span><br><span class="line">                    <span class="string">&quot;sZeroRecords&quot;</span>: <span class="string">&quot;没有匹配结果&quot;</span>,</span><br><span class="line">                    <span class="string">&quot;sInfo&quot;</span>: <span class="string">&quot;显示第 _START_ 至 _END_ 条结果，共 _TOTAL_ 条&quot;</span>,</span><br><span class="line">                    <span class="string">&quot;sInfoEmpty&quot;</span>: <span class="string">&quot;没有数据&quot;</span>,</span><br><span class="line">                    <span class="string">&quot;sInfoFiltered&quot;</span>: <span class="string">&quot;(获取 _MAX_ 条客户档案)&quot;</span>,</span><br><span class="line">                    <span class="string">&quot;sInfoPostFix&quot;</span>: <span class="string">&quot;&quot;</span>,</span><br><span class="line">                    <span class="string">&quot;sSearch&quot;</span>: <span class="string">&quot;搜索:&quot;</span>,</span><br><span class="line">                    <span class="string">&quot;sUrl&quot;</span>: <span class="string">&quot;&quot;</span>,</span><br><span class="line">                    <span class="string">&quot;sEmptyTable&quot;</span>: <span class="string">&quot;表中数据为空&quot;</span>,</span><br><span class="line">                    <span class="string">&quot;sLoadingRecords&quot;</span>: <span class="string">&quot;载入中...&quot;</span>,</span><br><span class="line">                    <span class="string">&quot;sInfoThousands&quot;</span>: <span class="string">&quot;,&quot;</span>,</span><br><span class="line">                    <span class="string">&quot;oPaginate&quot;</span>: &#123;</span><br><span class="line">                        <span class="string">&quot;sFirst&quot;</span>: <span class="string">&quot;首页&quot;</span>,</span><br><span class="line">                        <span class="string">&quot;sPrevious&quot;</span>: <span class="string">&quot;上页&quot;</span>,</span><br><span class="line">                        <span class="string">&quot;sNext&quot;</span>: <span class="string">&quot;下页&quot;</span>,</span><br><span class="line">                        <span class="string">&quot;sLast&quot;</span>: <span class="string">&quot;末页&quot;</span></span><br><span class="line">                    &#125;,</span><br><span class="line">                &#125;,</span><br><span class="line">            &#125;</span><br><span class="line">        );</span><br><span class="line">    &#125;</span><br><span class="line">&lt;/script&gt;</span><br></pre></td></tr></table></figure><p>5,6是个常见的条件格式问题，要用设置里的<strong>columnDefs参数解决，具体使用时主要涉及target和createdCell两个参数</strong>：</p><figure class="highlight js"><table><tr><td class="code"><pre><span class="line">&lt;script&gt;</span><br><span class="line">    <span class="keyword">function</span> <span class="title function_">initTable</span>(<span class="params">table</span>) &#123;</span><br><span class="line">        table.<span class="title class_">DataTable</span>(</span><br><span class="line">            &#123;</span><br><span class="line">                ...</span><br><span class="line">                <span class="attr">columnDefs</span>: [</span><br><span class="line">                    ...</span><br><span class="line">                    &#123;</span><br><span class="line">                        <span class="string">&quot;targets&quot;</span>: <span class="number">6</span>, <span class="comment">// 指定第7列EI</span></span><br><span class="line">                        <span class="string">&quot;createdCell&quot;</span>: <span class="keyword">function</span> (<span class="params">td, cellData, rowData, row, col</span>) &#123;</span><br><span class="line">                            <span class="keyword">if</span> (cellData &lt; <span class="number">100</span>) &#123;</span><br><span class="line">                                $(td).<span class="title function_">css</span>(<span class="string">&#x27;color&#x27;</span>, <span class="string">&#x27;red&#x27;</span>)</span><br><span class="line">                            &#125; <span class="keyword">else</span> <span class="keyword">if</span> (cellData &gt; <span class="number">100</span>) &#123;</span><br><span class="line">                                $(td).<span class="title function_">css</span>(<span class="string">&#x27;color&#x27;</span>, <span class="string">&#x27;green&#x27;</span>)</span><br><span class="line">                            &#125; <span class="keyword">else</span> <span class="keyword">if</span> (cellData.<span class="title function_">indexOf</span>(<span class="string">&quot;,&quot;</span>) !== -<span class="number">1</span>) &#123;</span><br><span class="line">                                $(td).<span class="title function_">css</span>(<span class="string">&#x27;color&#x27;</span>, <span class="string">&#x27;green&#x27;</span>)</span><br><span class="line">                            &#125;</span><br><span class="line">                        &#125;</span><br><span class="line">                    &#125;,</span><br><span class="line">                    &#123;</span><br><span class="line">                        <span class="string">&quot;targets&quot;</span>: [<span class="number">2</span>, <span class="number">4</span>, <span class="number">5</span>], <span class="comment">// 指定第3,5,6列绝对值变化，份额获取，增长率，这些有可能出现负数</span></span><br><span class="line">                        <span class="string">&quot;createdCell&quot;</span>: <span class="keyword">function</span> (<span class="params">td, cellData, rowData, row, col</span>) &#123;</span><br><span class="line">                            <span class="keyword">if</span> (cellData.<span class="title function_">startsWith</span>(<span class="string">&#x27;-&#x27;</span>)) &#123; <span class="comment">// 因为涉及到百分数的问题，这里用检查字符串的方法而不是&lt;0的方法判断负数</span></span><br><span class="line">                                $(td).<span class="title function_">css</span>(<span class="string">&#x27;color&#x27;</span>, <span class="string">&#x27;red&#x27;</span>)</span><br><span class="line">                            &#125;</span><br><span class="line">                        &#125;</span><br><span class="line">                    &#125;,</span><br><span class="line">                ]</span><br><span class="line">            &#125;</span><br><span class="line">        );</span><br><span class="line">    &#125;</span><br><span class="line">&lt;/script&gt;</span><br></pre></td></tr></table></figure><p>把份额列以条形图展示的方法来自一个叫percentageBars的插件，插件的初始化也是在columnDef参数里进行的：</p><figure class="highlight js"><table><tr><td class="code"><pre><span class="line">&lt;script&gt;</span><br><span class="line">    <span class="keyword">function</span> <span class="title function_">initTable</span>(<span class="params">table</span>) &#123;</span><br><span class="line">        table.<span class="title class_">DataTable</span>(</span><br><span class="line">            &#123;</span><br><span class="line">                ...</span><br><span class="line">                <span class="attr">columnDefs</span>: [</span><br><span class="line">                    &#123;<span class="string">&quot;width&quot;</span>: <span class="string">&quot;10%&quot;</span>, <span class="string">&quot;targets&quot;</span>: <span class="number">3</span>&#125;, <span class="comment">// 保持第4列份额列宽度固定，使条形图更美观</span></span><br><span class="line">                    &#123;</span><br><span class="line">                        <span class="attr">targets</span>: <span class="number">3</span>,</span><br><span class="line">                        <span class="attr">render</span>: $.fn.<span class="property">dataTable</span>.<span class="property">render</span>.<span class="title function_">percentBar</span>(<span class="string">&#x27;square&#x27;</span>, <span class="string">&#x27;#000&#x27;</span>, <span class="string">&#x27;#BCBCBC&#x27;</span>, <span class="string">&#x27;#00bfff&#x27;</span>, <span class="string">&#x27;#E6E6E6&#x27;</span>, <span class="number">1</span>, <span class="string">&#x27;ridge&#x27;</span>) <span class="comment">// 根据一定的色彩方案初始化条形图</span></span><br><span class="line">                    &#125;,</span><br><span class="line">                    ...</span><br><span class="line">                ]</span><br><span class="line">            &#125;</span><br><span class="line">        );</span><br><span class="line">    &#125;</span><br><span class="line">&lt;/script&gt;</span><br></pre></td></tr></table></figure><p>一键复制表格的功能我没有找到DataTables的内置方法，在display.html自行编写js配合前端按钮解决，注意html部分的ui top attached button和下面两个新的js：</p><figure class="highlight js"><table><tr><td class="code"><pre><span class="line">...</span><br><span class="line">&lt;div <span class="keyword">class</span>=<span class="string">&quot;ui tab segment&quot;</span> data-tab=<span class="string">&quot;competition&quot;</span>&gt;</span><br><span class="line">    <span class="language-xml"><span class="tag">&lt;<span class="name">h3</span> <span class="attr">class</span>=<span class="string">&quot;ui header&quot;</span>&gt;</span></span></span><br><span class="line"><span class="language-xml">        <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;content&quot;</span>&gt;</span></span></span><br><span class="line"><span class="language-xml">            最新横断面KPI一览</span></span><br><span class="line"><span class="language-xml">            <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;sub header&quot;</span>&gt;</span>数据表格<span class="tag">&lt;/<span class="name">div</span>&gt;</span></span></span><br><span class="line"><span class="language-xml">        <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span></span><br><span class="line"><span class="language-xml">    <span class="tag">&lt;/<span class="name">h3</span>&gt;</span></span></span><br><span class="line">    <span class="language-xml"><span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;ui divider&quot;</span>&gt;</span><span class="tag">&lt;/<span class="name">div</span>&gt;</span></span></span><br><span class="line">    <span class="language-xml"><span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;ui top attached button&quot;</span> <span class="attr">tabindex</span>=<span class="string">&quot;0&quot;</span></span></span></span><br><span class="line"><span class="tag"><span class="language-xml">         <span class="attr">onclick</span>=<span class="string">&quot;selectElementContents( document.getElementById(&#x27;ptable&#x27;) );&quot;</span></span></span></span><br><span class="line"><span class="tag"><span class="language-xml">         <span class="attr">data-content</span>=<span class="string">&quot;复制成功&quot;</span> <span class="attr">data-position</span>=<span class="string">&quot;bottom center&quot;</span>&gt;</span></span></span><br><span class="line"><span class="language-xml">        <span class="tag">&lt;<span class="name">i</span> <span class="attr">class</span>=<span class="string">&quot;copy icon&quot;</span>&gt;</span><span class="tag">&lt;/<span class="name">i</span>&gt;</span></span></span><br><span class="line"><span class="language-xml">        复制到剪贴板</span></span><br><span class="line"><span class="language-xml">    <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span></span><br><span class="line">    <span class="language-xml"><span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;ui hidden divider&quot;</span>&gt;</span><span class="tag">&lt;/<span class="name">div</span>&gt;</span></span></span><br><span class="line">    <span class="language-xml"><span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;ui container&quot;</span> <span class="attr">id</span>=<span class="string">&#x27;result_table&#x27;</span> <span class="attr">style</span>=<span class="string">&quot;width: 100%; overflow-x: scroll; overflow-y: hidden;&quot;</span>&gt;</span></span></span><br><span class="line"><span class="language-xml">        <span class="comment">&lt;!-- Django渲染html代码时需要加入|safe，保证html不会被自动转义 --&gt;</span></span></span><br><span class="line"><span class="language-xml">        &#123;<span class="symbol">&amp;#123;</span> ptable|safe <span class="symbol">&amp;#125;</span>&#125;</span></span><br><span class="line"><span class="language-xml">    <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span></span><br><span class="line">&lt;/div&gt;</span><br><span class="line">...</span><br><span class="line"></span><br><span class="line">&lt;script&gt;</span><br><span class="line">    <span class="comment">// 复制有node结构的文本区域</span></span><br><span class="line">    <span class="keyword">function</span> <span class="title function_">selectElementContents</span>(<span class="params">el</span>) &#123;</span><br><span class="line">        <span class="keyword">var</span> body = <span class="variable language_">document</span>.<span class="property">body</span>, range, sel;</span><br><span class="line">        <span class="keyword">if</span> (<span class="variable language_">document</span>.<span class="property">createRange</span> &amp;&amp; <span class="variable language_">window</span>.<span class="property">getSelection</span>) &#123;</span><br><span class="line">            range = <span class="variable language_">document</span>.<span class="title function_">createRange</span>();</span><br><span class="line">            sel = <span class="variable language_">window</span>.<span class="title function_">getSelection</span>();</span><br><span class="line">            sel.<span class="title function_">removeAllRanges</span>();</span><br><span class="line">            <span class="keyword">try</span> &#123;</span><br><span class="line">                range.<span class="title function_">selectNodeContents</span>(el);</span><br><span class="line">                sel.<span class="title function_">addRange</span>(range);</span><br><span class="line">            &#125; <span class="keyword">catch</span> (e) &#123;</span><br><span class="line">                range.<span class="title function_">selectNode</span>(el);</span><br><span class="line">                sel.<span class="title function_">addRange</span>(range);</span><br><span class="line">            &#125;</span><br><span class="line">        &#125; <span class="keyword">else</span> <span class="keyword">if</span> (body.<span class="property">createTextRange</span>) &#123;</span><br><span class="line">            range = body.<span class="title function_">createTextRange</span>();</span><br><span class="line">            range.<span class="title function_">moveToElementText</span>(el);</span><br><span class="line">            range.<span class="title function_">select</span>();</span><br><span class="line">        &#125;</span><br><span class="line">        <span class="variable language_">document</span>.<span class="title function_">execCommand</span>(<span class="string">&quot;Copy&quot;</span>);</span><br><span class="line">    &#125;</span><br><span class="line">&lt;/script&gt;</span><br><span class="line"></span><br><span class="line"><span class="language-xml"><span class="tag">&lt;<span class="name">script</span>&gt;</span><span class="language-javascript"></span></span></span><br><span class="line"><span class="language-javascript"><span class="language-xml">    <span class="comment">// 按钮点击后有弹出文本，显示data-content内容“复制成功”</span></span></span></span><br><span class="line"><span class="language-javascript"><span class="language-xml">    $(<span class="string">&#x27;.ui.top.attached.button&#x27;</span>)</span></span></span><br><span class="line"><span class="language-javascript"><span class="language-xml">        .<span class="title function_">popup</span>(&#123;</span></span></span><br><span class="line"><span class="language-javascript"><span class="language-xml">            <span class="attr">on</span>: <span class="string">&#x27;click&#x27;</span></span></span></span><br><span class="line"><span class="language-javascript"><span class="language-xml">        &#125;)</span></span></span><br><span class="line"><span class="language-javascript"><span class="language-xml">    ;</span></span></span><br><span class="line"><span class="language-javascript"><span class="language-xml"></span><span class="tag">&lt;/<span class="name">script</span>&gt;</span></span></span><br></pre></td></tr></table></figure><p>一个用户体验非常完善的表格就完成了：<br><img src="/images/python-djangosqlpand/v2-9c58782b3549f69288a303b2e48f6275_1440w.webp"></p>]]>
    </content>
    <id>https://blog.malu.tech/python-django-sql-pandas-pyecharts-data-analysis-platform-8/</id>
    <link href="https://blog.malu.tech/python-django-sql-pandas-pyecharts-data-analysis-platform-8/"/>
    <published>2024-02-18T16:00:00.000Z</published>
    <summary>Python Django+SQL+Pandas+Pyecharts自建在线数据分析平台系列文章第八篇：DataTables接管前端表格</summary>
    <title>Python Django+SQL+Pandas+Pyecharts自建在线数据分析平台（八）</title>
    <updated>2024-02-18T16:00:00.000Z</updated>
  </entry>
  <entry>
    <author>
      <name>Road</name>
    </author>
    <category term="Python" scheme="https://blog.malu.tech/categories/Python/"/>
    <category term="Django" scheme="https://blog.malu.tech/categories/Python/Django/"/>
    <category term="数据分析" scheme="https://blog.malu.tech/categories/Python/Django/%E6%95%B0%E6%8D%AE%E5%88%86%E6%9E%90/"/>
    <category term="Python" scheme="https://blog.malu.tech/tags/Python/"/>
    <category term="Django" scheme="https://blog.malu.tech/tags/Django/"/>
    <category term="SQL" scheme="https://blog.malu.tech/tags/SQL/"/>
    <category term="Pandas" scheme="https://blog.malu.tech/tags/Pandas/"/>
    <category term="Pyecharts" scheme="https://blog.malu.tech/tags/Pyecharts/"/>
    <category term="数据分析平台" scheme="https://blog.malu.tech/tags/%E6%95%B0%E6%8D%AE%E5%88%86%E6%9E%90%E5%B9%B3%E5%8F%B0/"/>
    <content>
      <![CDATA[<blockquote><p><strong>文章来源</strong>：<a href="https://zhuanlan.zhihu.com/p/142490087">Python Django+SQL+Pandas+Pyecharts自建在线数据分析平台（一）</a><br><strong>作者</strong>：<a href="https://www.zhihu.com/people/chen-cheng-76-40">ccpic</a><br><strong>感谢</strong>：感谢作者 ccpic 分享的优质内容，本网页主要用于学习知识的存档备份，欢迎点击原网页支持作者。</p></blockquote><p>（一）需求分析&amp;技术实现</p><p>（二）初步搭建Django环境</p><p>（三）页面布局&amp;Django模板</p><p>（四）SQL+Pandas初步处理数据</p><p>（五）前端表单交互</p><p>（六）Ajax异步传参与加载</p><h2 id="（七）前端数据格式的处理"><a href="#（七）前端数据格式的处理" class="headerlink" title="（七）前端数据格式的处理"></a>（七）前端数据格式的处理</h2><p>（八）DataTables接管前端表格</p><p>（九）Pyecharts实现交互图表</p><p>（十）静态图表的展示</p><p>（十一）“导出数据至Excel”功能</p><p>（十二）添加和配置缓存</p><p>（十三）用户登录系统</p><p>（十四）部署Django至生产环境</p><p>从本章开始，系列文章进入数据可视化的部分，我们从一个比较小的话题前端的数据格式开始。</p><p>其实本章的位置在其他任何Django系列教程里是要留给**自定义tag和filter（标签筛选器）**大谈特谈的，但是我们的项目舍弃了Django ORM又大量使用了AJAX，自然前端模板很难用到渲染的tag，tag filter也就无用武之地了。</p><p>但还是简单介绍下，毕竟这也是Django的特色之一。我们在前端其实还是用过一次tag filter的，是在第五章前端利用字典循环创建多选框时，有如下语句：</p><figure class="highlight html"><table><tr><td class="code"><pre><span class="line"><span class="tag">&lt;<span class="name">select</span> <span class="attr">name</span>=<span class="string">&quot;&#123;<span class="symbol">&amp;#123;</span> value.select|add:&quot;</span><span class="attr">_select</span>[]&quot; &amp;#<span class="attr">125</span>;&#125;&quot; <span class="attr">id</span>=<span class="string">&quot;&#123;<span class="symbol">&amp;#123;</span> value.select|add:&quot;</span><span class="attr">_select</span>&quot; &amp;#<span class="attr">125</span>;&#125;&quot; <span class="attr">multiple</span>=<span class="string">&quot;&quot;</span></span></span><br><span class="line"><span class="tag">                            <span class="attr">class</span>=<span class="string">&quot;ui fluid search dropdown&quot;</span>&gt;</span></span><br></pre></td></tr></table></figure><p>其中的双大括号部分，就是一个tag+filter的组合，竖杠后面的部分是竖杠前方tag的filter，冒号后面部分则为这个filter的参数：</p><figure class="highlight django"><table><tr><td class="code"><pre><span class="line"><span class="language-xml">&#123;<span class="symbol">&amp;#123;</span> value.select|add:&quot;_select&quot; <span class="symbol">&amp;#125;</span>&#125;</span></span><br></pre></td></tr></table></figure><p>这个语句的作用为，对value字典里的select的值与”_select”做字符串拼接。</p><p>可以看出filter在Django里的定位是：</p><ul><li><strong>本质上是一种函数</strong></li><li><strong>用于修改后端数据在前端的呈现方式</strong></li><li><strong>不对数据做任何永久修改</strong></li><li><strong>任务相对简单直接</strong></li></ul><p>这个定位和数据分析平台对数据格式展示的严格追求是不谋而合的，又比较简洁有效。在有条件应用Django tag filter时应该尽可能地使用它，尤其当Django自带的filter无法满足你的时候，可以自定义filter。</p><p>本章我们的目标之一是给后台传来的market_gr变量在前端显示时转换为“保留一位小数的百分号”格式。假设我们在前端有一个{&#123; market_gr &#125;}的tag占位符，我们可以自定义一个filter轻易做到这一点。</p><p>首先，在app文件夹下和templates平行的目录创建一个文件夹templatetags，并在该文件夹中新建两个.py文件——__init__.py和tags.py。此时的项目文件夹结构应该变成了这样：<br><img src="/images/python-djangosqlpand/v2-f8a1cee107719df6d51f29becf05e2a0_1440w.webp"></p><p>再修改工程settings.py文件<br><img src="/images/python-djangosqlpand/v2-9f90995c75f1e5e169d6819f3b1d1e2a_1440w.webp"></p><p>在TEMPLATE.OPTIONS里加入libraries参数，并指向之前创建的templatetags文件夹内的tags.py：</p><figure class="highlight text"><table><tr><td class="code"><pre><span class="line">&#x27;libraries&#x27;: &#123;</span><br><span class="line">    &#x27;tags&#x27;: &#x27;chpa_data.templatetags.tags&#x27;,</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>回到templatetags文件夹，其中__init__.py文件确保目录被视为一个Python包，可以保持为空。在tags.py里编写我们需要的filter方法。比如一个把数字转化为百分号的函数：</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">from</span> django <span class="keyword">import</span> template</span><br><span class="line"></span><br><span class="line">register = template.Library()</span><br><span class="line"></span><br><span class="line"><span class="meta">@register.filter(<span class="params">name=<span class="string">&#x27;percentage&#x27;</span></span>)</span></span><br><span class="line"><span class="keyword">def</span> <span class="title function_">percentage</span>(<span class="params">value, decimal</span>):</span><br><span class="line">    <span class="keyword">try</span>:</span><br><span class="line">        format_str = <span class="string">&#x27;&#123;0:.&#x27;</span>+ <span class="built_in">str</span>(decimal) + <span class="string">&#x27;&amp;#37;&#125;&#x27;</span></span><br><span class="line">        <span class="keyword">return</span> format_str.<span class="built_in">format</span>(value)</span><br><span class="line">    <span class="keyword">except</span>:</span><br><span class="line">        <span class="keyword">return</span> value</span><br></pre></td></tr></table></figure><p>此时把前端模板文件里要加入：</p><figure class="highlight django"><table><tr><td class="code"><pre><span class="line"><span class="language-xml">&#123;<span class="symbol">&amp;#37;</span> load tags <span class="symbol">&amp;#37;</span>&#125;</span></span><br></pre></td></tr></table></figure><p>再把</p><figure class="highlight text"><table><tr><td class="code"><pre><span class="line">&#123;&amp;#123; market_gr &amp;#125;&#125;</span><br></pre></td></tr></table></figure><p>改写成</p><figure class="highlight django"><table><tr><td class="code"><pre><span class="line"><span class="language-xml">&#123;<span class="symbol">&amp;#123;</span> market_gr|percentage:1 <span class="symbol">&amp;#125;</span>&#125;</span></span><br></pre></td></tr></table></figure><p>即可在前端显示时把任意float以“保留一位小数的百分号”的格式显示。</p><p><strong>除了自定义filter，同时也推荐大家试试看Django自带的humanize包内置的filters，这里不展开了~</strong></p><p>这时候问题来了，我的market_gr在现在的版本是用AJAX异步加载的返回里调用的，没法应用Django tag。这时教条地说，<strong>应该考虑前后分离的问题，在前端用JS完成数据的格式化。主要因为涉及到数据复用但格式不同的问题。</strong></p><p>实际上我尝试了2种做法。</p><p>JS转换为百分数的例子，编辑前端filter.html模板：</p><figure class="highlight js"><table><tr><td class="code"><pre><span class="line">&lt;script type=<span class="string">&quot;text/javascript&quot;</span>&gt;</span><br><span class="line">    $(<span class="string">&quot;#AJAX_get&quot;</span>).<span class="title function_">click</span>(<span class="keyword">function</span> (<span class="params">event</span>) &#123;</span><br><span class="line">        ...</span><br><span class="line">        $.<span class="title function_">ajax</span>(&#123;</span><br><span class="line">            ...</span><br><span class="line">            <span class="attr">success</span>: <span class="keyword">function</span> (<span class="params">ret</span>) &#123;     <span class="comment">//成功执行</span></span><br><span class="line">                ...</span><br><span class="line">                $(<span class="string">&quot;#value_gr&quot;</span>).<span class="title function_">html</span>(<span class="title function_">toPercent</span>(ret[<span class="string">&quot;market_gr&quot;</span>]));</span><br><span class="line">                <span class="comment">// 根据返回数据着色</span></span><br><span class="line">                <span class="keyword">if</span> (ret[<span class="string">&quot;market_gr&quot;</span>] &lt; <span class="number">0</span>)&#123;</span><br><span class="line">                    $(<span class="string">&quot;#div_gr&quot;</span>).<span class="title function_">removeClass</span>().<span class="title function_">addClass</span>(<span class="string">&quot;red statistic&quot;</span>);</span><br><span class="line">                &#125; <span class="keyword">else</span> <span class="keyword">if</span> (ret[<span class="string">&quot;market_gr&quot;</span>] &gt; <span class="number">0</span>) &#123;</span><br><span class="line">                    $(<span class="string">&quot;#div_gr&quot;</span>).<span class="title function_">removeClass</span>().<span class="title function_">addClass</span>(<span class="string">&quot;green statistic&quot;</span>);</span><br><span class="line">                &#125;</span><br><span class="line">            &#125;,</span><br><span class="line">            ...</span><br><span class="line">        &#125;);</span><br><span class="line">    &#125;)</span><br><span class="line"></span><br><span class="line">    <span class="keyword">function</span> <span class="title function_">toPercent</span>(<span class="params">str</span>)&#123;</span><br><span class="line">        <span class="keyword">if</span> (<span class="built_in">isNaN</span>(str) === <span class="literal">false</span>)&#123;</span><br><span class="line">            <span class="keyword">var</span> strP=<span class="title class_">Number</span>(str*<span class="number">100</span>).<span class="title function_">toFixed</span>(<span class="number">1</span>);</span><br><span class="line">            strP+=<span class="string">&quot;%&quot;</span>;</span><br><span class="line">        &#125; <span class="keyword">else</span> &#123;</span><br><span class="line">            strP = str;</span><br><span class="line">        &#125;</span><br><span class="line">        <span class="keyword">return</span> strP;</span><br><span class="line">    &#125;</span><br><span class="line">&lt;/script&gt;</span><br></pre></td></tr></table></figure><p>后端views.py转换百分数的例子：</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">def</span> <span class="title function_">kpi</span>(<span class="params">df</span>):</span><br><span class="line">    ...</span><br><span class="line">        </span><br><span class="line">    <span class="keyword">return</span> [market_size, <span class="string">&quot;&#123;0:.1&amp;#37;&#125;&quot;</span>.<span class="built_in">format</span>(market_gr), <span class="string">&quot;&#123;0:.1&amp;#37;&#125;&quot;</span>.<span class="built_in">format</span>(market_cagr)]</span><br></pre></td></tr></table></figure><p>这里可以很明显看出后端调整格式的优缺点，优点是可以利用Python语法的便利性一行代码解决问题。缺点是输出的类型有可能变成字符串，导致前端后续一些操作更加麻烦（比如上方代码块中根据返回数据判断的条件语句）。而前端一些js操作html的便利性又是后端无法代替的。</p><p>但有时后端修改格式也是必须的，比如我们的表格ptable是用整个df.to_html()直接渲染的，前端修改格式就会麻烦得多。</p><p>此时有一种利用df.to_html的formatters参数专门为表格调整数据格式的快速方式，且该方法可以复用到很多其他的pandas表格输出结果上。</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">def</span> <span class="title function_">query</span>(<span class="params">request</span>):</span><br><span class="line">    ...</span><br><span class="line">    table = ptable(pivoted)</span><br><span class="line">    table = table.to_html(formatters=build_formatters_by_col(table))</span><br><span class="line"></span><br><span class="line">    context = &#123;</span><br><span class="line">        ...</span><br><span class="line">        <span class="string">&#x27;ptable&#x27;</span>: table,</span><br><span class="line">    &#125;</span><br><span class="line">    ...</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="keyword">def</span> <span class="title function_">build_formatters_by_col</span>(<span class="params">df</span>):</span><br><span class="line">     format_abs = <span class="keyword">lambda</span> x: <span class="string">&#x27;&#123;:,.0f&#125;&#x27;</span>.<span class="built_in">format</span>(x)</span><br><span class="line">     format_share = <span class="keyword">lambda</span> x: <span class="string">&#x27;&#123;:.1&amp;#37;&#125;&#x27;</span>.<span class="built_in">format</span>(x)</span><br><span class="line">     format_gr = <span class="keyword">lambda</span> x: <span class="string">&#x27;&#123;:.1&amp;#37;&#125;&#x27;</span>.<span class="built_in">format</span>(x)</span><br><span class="line">     format_currency = <span class="keyword">lambda</span> x: <span class="string">&#x27;¥&#123;:,.0f&#125;&#x27;</span>.<span class="built_in">format</span>(x)</span><br><span class="line">     d = &#123;&#125;</span><br><span class="line">     <span class="keyword">for</span> column <span class="keyword">in</span> df.columns:</span><br><span class="line">          <span class="keyword">if</span> <span class="string">&#x27;份额&#x27;</span> <span class="keyword">in</span> column <span class="keyword">or</span> <span class="string">&#x27;贡献&#x27;</span> <span class="keyword">in</span> column:</span><br><span class="line">               d[column] = format_share</span><br><span class="line">          <span class="keyword">elif</span> <span class="string">&#x27;价格&#x27;</span> <span class="keyword">in</span> column <span class="keyword">or</span> <span class="string">&#x27;单价&#x27;</span> <span class="keyword">in</span> column:</span><br><span class="line">               d[column] = format_currency</span><br><span class="line">          <span class="keyword">elif</span> <span class="string">&#x27;同比增长&#x27;</span> <span class="keyword">in</span> column <span class="keyword">or</span> <span class="string">&#x27;增长率&#x27;</span> <span class="keyword">in</span> column <span class="keyword">or</span> <span class="string">&#x27;CAGR&#x27;</span> <span class="keyword">in</span> column <span class="keyword">or</span> <span class="string">&#x27;同比变化&#x27;</span> <span class="keyword">in</span> column:</span><br><span class="line">               d[column] = format_gr</span><br><span class="line">          <span class="keyword">else</span>:</span><br><span class="line">               d[column] = format_abs</span><br><span class="line">     <span class="keyword">return</span> d</span><br></pre></td></tr></table></figure><p>此时的前端表格已经格式化了：<br><img src="/images/python-djangosqlpand/v2-c956893b08c9f9fde76f437087fd5cd5_1440w.webp"></p><p>而要达到上方20行简单Python语句就能达到的效果，用JS重构必须付出数倍的努力。</p><p>所以这些涉及展示的非核心问题，在不是硬性要求前后端分离的场合，我最后的结论是——就八仙过海各显神通好了。</p>]]>
    </content>
    <id>https://blog.malu.tech/python-django-sql-pandas-pyecharts-data-analysis-platform-7/</id>
    <link href="https://blog.malu.tech/python-django-sql-pandas-pyecharts-data-analysis-platform-7/"/>
    <published>2024-02-11T16:00:00.000Z</published>
    <summary>Python Django+SQL+Pandas+Pyecharts自建在线数据分析平台系列文章第七篇：前端数据格式的处理</summary>
    <title>Python Django+SQL+Pandas+Pyecharts自建在线数据分析平台（七）</title>
    <updated>2024-02-11T16:00:00.000Z</updated>
  </entry>
  <entry>
    <author>
      <name>Road</name>
    </author>
    <category term="Python" scheme="https://blog.malu.tech/categories/Python/"/>
    <category term="Django" scheme="https://blog.malu.tech/categories/Python/Django/"/>
    <category term="数据分析" scheme="https://blog.malu.tech/categories/Python/Django/%E6%95%B0%E6%8D%AE%E5%88%86%E6%9E%90/"/>
    <category term="Python" scheme="https://blog.malu.tech/tags/Python/"/>
    <category term="Django" scheme="https://blog.malu.tech/tags/Django/"/>
    <category term="SQL" scheme="https://blog.malu.tech/tags/SQL/"/>
    <category term="Pandas" scheme="https://blog.malu.tech/tags/Pandas/"/>
    <category term="Pyecharts" scheme="https://blog.malu.tech/tags/Pyecharts/"/>
    <category term="数据分析平台" scheme="https://blog.malu.tech/tags/%E6%95%B0%E6%8D%AE%E5%88%86%E6%9E%90%E5%B9%B3%E5%8F%B0/"/>
    <content>
      <![CDATA[<blockquote><p><strong>文章来源</strong>：<a href="https://zhuanlan.zhihu.com/p/142490087">Python Django+SQL+Pandas+Pyecharts自建在线数据分析平台（一）</a><br><strong>作者</strong>：<a href="https://www.zhihu.com/people/chen-cheng-76-40">ccpic</a><br><strong>感谢</strong>：感谢作者 ccpic 分享的优质内容，本网页主要用于学习知识的存档备份，欢迎点击原网页支持作者。</p></blockquote><p>（一）需求分析&amp;技术实现</p><p>（二）初步搭建Django环境</p><p>（三）页面布局&amp;Django模板</p><p>（四）SQL+Pandas初步处理数据</p><p>（五）前端表单交互</p><h2 id="（六）Ajax异步传参与加载"><a href="#（六）Ajax异步传参与加载" class="headerlink" title="（六）Ajax异步传参与加载"></a>（六）Ajax异步传参与加载</h2><p>（七）前端数据格式的处理</p><p>（八）DataTables接管前端表格</p><p>（九）Pyecharts实现交互图表</p><p>（十）静态图表的展示</p><p>（十一）“导出数据至Excel”功能</p><p>（十二）添加和配置缓存</p><p>（十三）用户登录系统</p><p>（十四）部署Django至生产环境</p><p>在上一章中，我们已经准备好了前端的交互表单，我们期望提交表单后查询参数传回后台并返回相应的结果在前端展示。</p><p>在Django中，这种传参的方式是通过URL Dispatcher进行的，也即是把参数埋在关联后台方法的url里，实际通信时再解析出来作为View中Python方法的参数。我们在上章中其实已经有过定义这种包含参数的动态URL的经验，URL中的parameter（Django文档里称为captured value）用下方这种括号格式表示，括号内冒号前的内容为指定的参数类型转化器，默认为str（不包含’&#x2F;‘的字符串）。</p><p>下面的URL即为捕获2个字符串参数column和kw：</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line">urlpatterns = [</span><br><span class="line">    ...</span><br><span class="line">    path(<span class="string">r&#x27;search/&lt;str:column&gt;/&lt;str:kw&gt;&#x27;</span>, views.search, name=<span class="string">&#x27;search&#x27;</span>)</span><br><span class="line">]</span><br></pre></td></tr></table></figure><p>所以这时我们会朴素地想到，我们建一个长长的URL囊括所有前端参数，不就可以传参了。Submit按钮只需要根据表单选择结果负责跳转到相应的URL就可以了。</p><p>这么做确实是可以的，但我们需要知道，<strong>避免重定向是提高网页性能和用户体验的非常重要的一条原则，而这里同时还有个同步加载和异步加载的问题。</strong></p><p>异步加载的性能优势很多人喜欢用类似下面的示意图解释，最终节省了大量的加载时间。<br><img src="/images/python-djangosqlpand/v2-f5032ddfaf07a7fbd47624a79ab843dd_1440w.webp"></p><p>作为一个应用层面的二手程序员，在本例的场景中我更倾向于把异步加载在传参中的应用主要看成一个前端优化。传统的同步加载一般会根据表单参数返回一个完整的网页，但异步加载仅向服务器发送并取回必须的数据，并在不刷新网页的情况下在局部无缝加载。</p><p>其实在上章中我们已经不知不觉完成了一次异步加载，表单下拉框备选项的服务器响应其实就是一次异步加载。但因为封装在Semantic UI的API中，不用我们操心，现在需要我们用jQuery自己实现一次。</p><p>本例中我们使用AJAX（异步的JavaScript和XML技术）实现异步传参，实际操作中AJAX的传参有2种请求方式，一般认为GET在涉及传输和缓存的性能上都更具优势：</p><ul><li>GET</li><li>POST</li></ul><p>数据也可以有多种形式，如：</p><ul><li>数组&#x2F;字典</li><li>Json</li><li>表单序列化，结果类似Json</li></ul><p>jQuery语法上也分为两种写法：</p><ul><li>通过URL传参（类似Django默认方法）</li><li>通过data传参</li></ul><p>本例中我们使用最通用的通过GET用data参数传递字典的方法。我们在上章中放置了一个名为”AJAX_get”的“筛选”提交按钮，编写这个按钮的click function如下：</p><figure class="highlight js"><table><tr><td class="code"><pre><span class="line">&lt;script type=<span class="string">&quot;text/javascript&quot;</span>&gt;</span><br><span class="line">    $(<span class="string">&quot;#AJAX_get&quot;</span>).<span class="title function_">click</span>(<span class="keyword">function</span> (<span class="params">event</span>) &#123;</span><br><span class="line">        event.<span class="title function_">preventDefault</span>(); <span class="comment">// 防止表单默认的提交</span></span><br><span class="line">        <span class="comment">// 获取单选下拉框的值</span></span><br><span class="line">        <span class="keyword">var</span> form_data = &#123;</span><br><span class="line">            <span class="string">&quot;DIMENSION_select&quot;</span>: $(<span class="string">&quot;#DIMENSION_select&quot;</span>).<span class="title function_">val</span>(),</span><br><span class="line">            <span class="string">&quot;PERIOD_select&quot;</span>: $(<span class="string">&quot;#PERIOD_select&quot;</span>).<span class="title function_">val</span>(),</span><br><span class="line">            <span class="string">&quot;UNIT_select&quot;</span>: $(<span class="string">&quot;#UNIT_select&quot;</span>).<span class="title function_">val</span>(),</span><br><span class="line">        &#125;;</span><br><span class="line"></span><br><span class="line">        <span class="comment">// 获取多选下拉框的值</span></span><br><span class="line">        <span class="keyword">var</span> dict = &#123;&amp;#<span class="number">123</span>; mselect_dict|safe &amp;#<span class="number">125</span>;&#125;;</span><br><span class="line">        <span class="keyword">for</span> (key <span class="keyword">in</span> dict) &#123;</span><br><span class="line">            <span class="keyword">var</span> form_name = dict[key][<span class="string">&#x27;select&#x27;</span>] + <span class="string">&quot;_select&quot;</span>;</span><br><span class="line">            jquery_selector_id = <span class="string">&quot;[id=&#x27;&quot;</span> + form_name + <span class="string">&quot;&#x27;]&quot;</span>;<span class="comment">//因为我们的部分多选框id有空格，要用这种写法</span></span><br><span class="line">            form_data[form_name] = $(jquery_selector_id).<span class="title function_">val</span>();</span><br><span class="line">        &#125;</span><br><span class="line"></span><br><span class="line">        $.<span class="title function_">ajax</span>(&#123;</span><br><span class="line">            <span class="comment">// 请求的url</span></span><br><span class="line">            <span class="attr">url</span>: <span class="string">&#x27;&#123;&amp;#37; url &#x27;</span><span class="attr">chpa</span>:query<span class="string">&#x27; &amp;#37;&#125;&#x27;</span>,</span><br><span class="line">            <span class="comment">// 请求的type</span></span><br><span class="line">            <span class="attr">type</span>: <span class="string">&#x27;GET&#x27;</span>,</span><br><span class="line">            <span class="comment">// 发送的数据</span></span><br><span class="line">            <span class="attr">data</span>: form_data,</span><br><span class="line">            <span class="comment">// 回调函数，其中ret是返回的JSON，可以以字典的方式调用</span></span><br><span class="line">            <span class="attr">success</span>: <span class="keyword">function</span> (<span class="params">ret</span>) &#123;     <span class="comment">//成功执行</span></span><br><span class="line"></span><br><span class="line">            &#125;,</span><br><span class="line">            <span class="attr">error</span>: <span class="keyword">function</span> (<span class="params"></span>) &#123;            <span class="comment">//失败</span></span><br><span class="line">                <span class="variable language_">console</span>.<span class="title function_">log</span>(<span class="string">&#x27;失败&#x27;</span>)</span><br><span class="line">            &#125;</span><br><span class="line">        &#125;);</span><br><span class="line">    &#125;)</span><br><span class="line">&lt;/script&gt;</span><br></pre></td></tr></table></figure><p>这里别的部分都一目了然，要特别注意下面这一句，没有此语句Django后台会在每次提交时报错，虽然不影响使用，但是很烦人：</p><figure class="highlight text"><table><tr><td class="code"><pre><span class="line">event.preventDefault(); // 防止表单默认的提交</span><br></pre></td></tr></table></figure><p>我们需要在url.py编辑上方代码块中$.ajax部分对应的url，新建一个query的URL pattern，并绑定到views.py中的query方法：</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line">urlpatterns = [</span><br><span class="line">    ...</span><br><span class="line">    path(<span class="string">r&#x27;query&#x27;</span>, views.query, name=<span class="string">&#x27;query&#x27;</span>),</span><br><span class="line">]</span><br></pre></td></tr></table></figure><p>此时如果前端点击筛选按钮，我们已经可以通过浏览器的扩展工具或者观察后端的request.GET变量看到，表单的选择项已经发送到后端了。<br><img src="/images/python-djangosqlpand/v2-20b1fda9a074b0706c09161fb0e8a1e9_1440w.webp"></p><p><img src="/images/python-djangosqlpand/v2-20b1fda9a074b0706c09161fb0e8a1e9_1440w.webp"></p><p>后端读取到request.GET的格式为QueryDict</p><p>那么在views.py中，我们的query方法要实现以下后续功能：</p><ol><li>解析前端参数到理想格式</li><li>根据前端参数数据拼接SQL并用Pandas读取</li><li>Pandas读取数据后，将前端选择的DIMENSION作为pivot_table方法的column参数</li><li>返回Json格式的结果</li></ol><p>步骤1最简单的方法就是用下面的语句把request.GET从QueryDict直接转化为Python字典，这个方法最快速，但得到的字典中即使是单选表单的value也是列表，需要注意。当然也可以使用request.GET.get和request.GET.getlist方法精细处理每个参数。</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">try</span>:</span><br><span class="line">    <span class="keyword">import</span> six  <span class="comment"># for modern Django</span></span><br><span class="line"><span class="keyword">except</span> ImportError:</span><br><span class="line">    <span class="keyword">from</span> django.utils <span class="keyword">import</span> six  <span class="comment"># for legacy Django</span></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="keyword">def</span> <span class="title function_">query</span>(<span class="params">request</span>):</span><br><span class="line">    ...</span><br><span class="line">    form_dict = <span class="built_in">dict</span>(six.iterlists(request.GET))</span><br><span class="line">    ...</span><br></pre></td></tr></table></figure><p>步骤2涉及到参数拼接的逻辑连接，本例中取简单的“AND”关系，但实际中有可能AND关系会不够用。这里未来有2种进步方式，1是在前端准备更为复杂的表单，纳入更多逻辑关系符号（甚至可能还需要包括括号位置），也就是实现一个前端的SQL条件语句编译器；2是偷懒的做法，额外提供一个文本框表单，直接传SQL条件语句到后方。不过这样也跟我们直接SQL提取数据做分析差不多了。</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">def</span> <span class="title function_">sqlparse</span>(<span class="params">context</span>):</span><br><span class="line">    <span class="built_in">print</span>(context)</span><br><span class="line">    sql = <span class="string">&quot;Select * from %s Where PERIOD = &#x27;%s&#x27; And UNIT = &#x27;%s&#x27;&quot;</span> % \</span><br><span class="line">          (DB_TABLE, context[<span class="string">&#x27;PERIOD_select&#x27;</span>][<span class="number">0</span>], context[<span class="string">&#x27;UNIT_select&#x27;</span>][<span class="number">0</span>])  <span class="comment"># 先处理单选部分</span></span><br><span class="line"></span><br><span class="line">    <span class="comment"># 下面循环处理多选部分</span></span><br><span class="line">    <span class="keyword">for</span> k, v <span class="keyword">in</span> context.items():</span><br><span class="line">        <span class="keyword">if</span> k <span class="keyword">not</span> <span class="keyword">in</span> [<span class="string">&#x27;csrfmiddlewaretoken&#x27;</span>, <span class="string">&#x27;DIMENSION_select&#x27;</span>, <span class="string">&#x27;PERIOD_select&#x27;</span>, <span class="string">&#x27;UNIT_select&#x27;</span>]:</span><br><span class="line">            field_name = k[:-<span class="number">9</span>]  <span class="comment"># 字段名</span></span><br><span class="line">            selected = v  <span class="comment"># 选择项</span></span><br><span class="line">            sql = sql_extent(sql, field_name, selected)  <span class="comment">#未来可以通过进一步拼接字符串动态扩展sql语句</span></span><br><span class="line">    <span class="keyword">return</span> sql</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="keyword">def</span> <span class="title function_">sql_extent</span>(<span class="params">sql, field_name, selected, operator=<span class="string">&quot; AND &quot;</span></span>):</span><br><span class="line">    <span class="keyword">if</span> selected <span class="keyword">is</span> <span class="keyword">not</span> <span class="literal">None</span>:</span><br><span class="line">        statement = <span class="string">&#x27;&#x27;</span></span><br><span class="line">        <span class="keyword">for</span> data <span class="keyword">in</span> selected:</span><br><span class="line">            statement = statement + <span class="string">&quot;&#x27;&quot;</span> + data + <span class="string">&quot;&#x27;, &quot;</span></span><br><span class="line">        statement = statement[:-<span class="number">2</span>]</span><br><span class="line">        <span class="keyword">if</span> statement != <span class="string">&#x27;&#x27;</span>:</span><br><span class="line">            sql = sql + operator + field_name + <span class="string">&quot; in (&quot;</span> + statement + <span class="string">&quot;)&quot;</span></span><br><span class="line">    <span class="keyword">return</span> sql</span><br></pre></td></tr></table></figure><p>步骤3就是把之前第四章处理数据的方法拿过来，参数变成动态的。</p><p>步骤4修改return语句返回的类型为Json，不渲染具体页面。</p><p>整个query方法最后差不多应该长这样：</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">def</span> <span class="title function_">query</span>(<span class="params">request</span>):</span><br><span class="line">    form_dict = <span class="built_in">dict</span>(six.iterlists(request.GET))</span><br><span class="line">    sql = sqlparse(form_dict)  <span class="comment"># sql拼接</span></span><br><span class="line">    <span class="built_in">print</span>(sql)</span><br><span class="line">    df = pd.read_sql_query(sql, ENGINE)  <span class="comment"># 将sql语句结果读取至Pandas Dataframe</span></span><br><span class="line"></span><br><span class="line">    dimension_selected = form_dict[<span class="string">&#x27;DIMENSION_select&#x27;</span>][<span class="number">0</span>]</span><br><span class="line">    <span class="comment">#  如果字段名有空格为了SQL语句在预设字典中加了中括号的，这里要去除</span></span><br><span class="line">    <span class="keyword">if</span> dimension_selected[<span class="number">0</span>] == <span class="string">&#x27;[&#x27;</span>:</span><br><span class="line"></span><br><span class="line">        column = dimension_selected[<span class="number">1</span>:][:-<span class="number">1</span>]</span><br><span class="line">    <span class="keyword">else</span>:</span><br><span class="line">        column = dimension_selected</span><br><span class="line">    pivoted = pd.pivot_table(df,</span><br><span class="line">                             values=<span class="string">&#x27;AMOUNT&#x27;</span>,  <span class="comment"># 数据透视汇总值为AMOUNT字段，一般保持不变</span></span><br><span class="line">                             index=<span class="string">&#x27;DATE&#x27;</span>,  <span class="comment"># 数据透视行为DATE字段，一般保持不变</span></span><br><span class="line">                             columns=column,  <span class="comment"># 数据透视列为前端选择的分析维度</span></span><br><span class="line">                             aggfunc=np.<span class="built_in">sum</span>)  <span class="comment"># 数据透视汇总方式为求和，一般保持不变</span></span><br><span class="line">    <span class="keyword">if</span> pivoted.empty <span class="keyword">is</span> <span class="literal">False</span>:</span><br><span class="line">        pivoted.sort_values(by=pivoted.index[-<span class="number">1</span>], axis=<span class="number">1</span>, ascending=<span class="literal">False</span>, inplace=<span class="literal">True</span>)  <span class="comment"># 结果按照最后一个DATE表现排序</span></span><br><span class="line"></span><br><span class="line">    <span class="comment"># KPI</span></span><br><span class="line">    kpi = get_kpi(pivoted)</span><br><span class="line"></span><br><span class="line">    context = &#123;</span><br><span class="line">        <span class="string">&quot;market_size&quot;</span>: kpi[<span class="string">&quot;market_size&quot;</span>],</span><br><span class="line">        <span class="string">&quot;market_gr&quot;</span>: kpi[<span class="string">&quot;market_gr&quot;</span>],</span><br><span class="line">        <span class="string">&quot;market_cagr&quot;</span>: kpi[<span class="string">&quot;market_cagr&quot;</span>],</span><br><span class="line">        <span class="string">&#x27;ptable&#x27;</span>: ptable(pivoted).to_html()</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="keyword">return</span> HttpResponse(json.dumps(context, ensure_ascii=<span class="literal">False</span>), content_type=<span class="string">&quot;application/json charset=utf-8&quot;</span>) <span class="comment"># 返回结果必须是json格式</span></span><br></pre></td></tr></table></figure><p>而index方法被解放了，只保留初始化表单备选项的功能：</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">def</span> <span class="title function_">index</span>(<span class="params">request</span>):</span><br><span class="line">    mselect_dict = &#123;&#125;</span><br><span class="line">    <span class="keyword">for</span> key, value <span class="keyword">in</span> D_MULTI_SELECT.items():</span><br><span class="line">        mselect_dict[key] = &#123;&#125;</span><br><span class="line">        mselect_dict[key][<span class="string">&#x27;select&#x27;</span>] = value</span><br><span class="line"></span><br><span class="line">    context = &#123;</span><br><span class="line">        <span class="string">&#x27;mselect_dict&#x27;</span>: mselect_dict</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="keyword">return</span> render(request, <span class="string">&#x27;chpa_data/display.html&#x27;</span>, context)</span><br></pre></td></tr></table></figure><p>我们此时已经可以测试这个异步传参的结果了，还记得在第四章我们实现的高血压药ARB的查询实例吗，它是静态的，通过后台人工输入的字符串作为参数查询。现在我们已经可以通过<a href="http://127.0.0.1:8088/chpa/query?%E5%8A%A8%E6%80%81%E6%9F%A5%E8%AF%A2%E7%BB%93%E6%9E%9C%E4%BA%86%E3%80%82">http://127.0.0.1:8088/chpa/query?动态查询结果了。</a></p><p>最后一个简单的步骤，在filter.html的JS代码里修改$.ajax部分的success参数，决定返回的结果怎么处理。我们选择把结果更新到之前模板预留id的DOM元素中。<br><img src="/images/python-djangosqlpand/v2-e268d044b7c4b445161852f7137b5b85_1440w.webp"></p><figure class="highlight js"><table><tr><td class="code"><pre><span class="line"><span class="attr">success</span>: <span class="keyword">function</span> (<span class="params">ret</span>) &#123;     <span class="comment">//成功执行</span></span><br><span class="line">    <span class="comment">// 更新单位标签</span></span><br><span class="line">    $(<span class="string">&quot;#label_size_unit&quot;</span>).<span class="title function_">html</span>(<span class="string">&quot;最新&quot;</span>+form_data[<span class="string">&#x27;PERIOD_select&#x27;</span>]+ <span class="string">&quot; &quot;</span> +form_data[<span class="string">&#x27;UNIT_select&#x27;</span>]);</span><br><span class="line">    <span class="comment">// 把查询结果输出到网页上预留id的DOM元素中</span></span><br><span class="line">    $(<span class="string">&quot;#value_size&quot;</span>).<span class="title function_">html</span>(ret[<span class="string">&quot;market_size&quot;</span>].<span class="title function_">toLocaleString</span>());</span><br><span class="line">    $(<span class="string">&quot;#value_gr&quot;</span>).<span class="title function_">html</span>(ret[<span class="string">&quot;market_gr&quot;</span>].<span class="title function_">toLocaleString</span>());</span><br><span class="line">    $(<span class="string">&quot;#value_cagr&quot;</span>).<span class="title function_">html</span>(ret[<span class="string">&quot;market_cagr&quot;</span>].<span class="title function_">toLocaleString</span>());</span><br><span class="line">    $(<span class="string">&quot;#result_table&quot;</span>).<span class="title function_">html</span>(ret[<span class="string">&#x27;ptable&#x27;</span>]);</span><br><span class="line">&#125;,</span><br></pre></td></tr></table></figure><p>这个项目的核心——网页前后端交互查询出数，可以说基本功能层面至此已经完成了~</p><p>最后还有些有关用户体验的细节问题，我们希望查询出数后有个loading界面，这在数据量大时很有用。可以使用Semantic UI的Dimmer（遮罩）组件，在Display.html的最前部分加入下面的语句：</p><figure class="highlight html"><table><tr><td class="code"><pre><span class="line">&#123;<span class="symbol">&amp;#37;</span> extends &quot;chpa_data/analysis.html&quot; <span class="symbol">&amp;#37;</span>&#125;</span><br><span class="line"></span><br><span class="line">&#123;<span class="symbol">&amp;#37;</span> block display <span class="symbol">&amp;#37;</span>&#125;</span><br><span class="line"><span class="comment">&lt;!-- 数据处理时的loading遮罩 --&gt;</span></span><br><span class="line"><span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;ui active dimmer&quot;</span> <span class="attr">id</span>=<span class="string">&quot;dimmer&quot;</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;ui text&quot;</span> <span class="attr">style</span>=<span class="string">&quot;color: #FFFFFF&quot;</span>&gt;</span>请使用左侧筛选框选择分析维度和定义市场<span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line"><span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">...</span><br></pre></td></tr></table></figure><p>然后在JS部分添加按钮点击过和AJAX成功回传过以后修改dimmer的语句：</p><figure class="highlight js"><table><tr><td class="code"><pre><span class="line">&lt;script type=<span class="string">&quot;text/javascript&quot;</span>&gt;</span><br><span class="line">    $(<span class="string">&quot;#AJAX_get&quot;</span>).<span class="title function_">click</span>(<span class="keyword">function</span> (<span class="params">event</span>) &#123;</span><br><span class="line">        ...</span><br><span class="line">        <span class="keyword">var</span> dimmer = $(<span class="string">&quot;#dimmer&quot;</span>);</span><br><span class="line">        dimmer.<span class="title function_">attr</span>(<span class="string">&#x27;class&#x27;</span>, <span class="string">&#x27;ui active dimmer&#x27;</span>); <span class="comment">// 点击筛选按钮后dimmer变成active</span></span><br><span class="line">        dimmer.<span class="title function_">children</span>(<span class="string">&#x27;div&#x27;</span>).<span class="title function_">remove</span>(); <span class="comment">// 删除初始化文字</span></span><br><span class="line">        dimmer.<span class="title function_">append</span>(<span class="string">&#x27;&lt;div class=&quot;ui text loader&quot;&gt;数据加载中……&lt;/div&gt;&#x27;</span>); <span class="comment">// 增加loading效果和文字</span></span><br><span class="line"></span><br><span class="line">        ...</span><br><span class="line">        $.<span class="title function_">ajax</span>(&#123;</span><br><span class="line">            ...</span><br><span class="line">            <span class="attr">success</span>: <span class="keyword">function</span> (<span class="params">ret</span>) &#123;     <span class="comment">//成功执行</span></span><br><span class="line">                <span class="comment">// 去除加载遮罩（去掉active）</span></span><br><span class="line">                dimmer.<span class="title function_">attr</span>(<span class="string">&#x27;class&#x27;</span>, <span class="string">&#x27;ui dimmer&#x27;</span>);</span><br><span class="line">                ...</span><br><span class="line">            &#125;,</span><br><span class="line">            <span class="attr">error</span>: <span class="keyword">function</span> (<span class="params"></span>) &#123;            <span class="comment">//失败</span></span><br><span class="line">                <span class="variable language_">console</span>.<span class="title function_">log</span>(<span class="string">&#x27;失败&#x27;</span>);</span><br><span class="line">                dimmer.<span class="title function_">children</span>(<span class="string">&#x27;div&#x27;</span>).<span class="title function_">text</span>(<span class="string">&#x27;有错误发生，无法完成查询&#x27;</span>); <span class="comment">// AJAX回调失败则报错</span></span><br><span class="line">            &#125;</span><br><span class="line">        &#125;);</span><br><span class="line">    &#125;)</span><br><span class="line">&lt;/script&gt;</span><br></pre></td></tr></table></figure><p>Loading界面就完成了：<br><img src="/images/python-djangosqlpand/v2-f101697c193bf098b41a2b6c13f71897_1440w.webp"></p>]]>
    </content>
    <id>https://blog.malu.tech/python-django-sql-pandas-pyecharts-data-analysis-platform-6/</id>
    <link href="https://blog.malu.tech/python-django-sql-pandas-pyecharts-data-analysis-platform-6/"/>
    <published>2024-02-04T16:00:00.000Z</published>
    <summary>Python Django+SQL+Pandas+Pyecharts自建在线数据分析平台系列文章第六篇：Ajax异步传参与加载</summary>
    <title>Python Django+SQL+Pandas+Pyecharts自建在线数据分析平台（六）</title>
    <updated>2024-02-04T16:00:00.000Z</updated>
  </entry>
  <entry>
    <author>
      <name>Road</name>
    </author>
    <category term="Python" scheme="https://blog.malu.tech/categories/Python/"/>
    <category term="Django" scheme="https://blog.malu.tech/categories/Python/Django/"/>
    <category term="数据分析" scheme="https://blog.malu.tech/categories/Python/Django/%E6%95%B0%E6%8D%AE%E5%88%86%E6%9E%90/"/>
    <category term="Python" scheme="https://blog.malu.tech/tags/Python/"/>
    <category term="Django" scheme="https://blog.malu.tech/tags/Django/"/>
    <category term="SQL" scheme="https://blog.malu.tech/tags/SQL/"/>
    <category term="Pandas" scheme="https://blog.malu.tech/tags/Pandas/"/>
    <category term="Pyecharts" scheme="https://blog.malu.tech/tags/Pyecharts/"/>
    <category term="数据分析平台" scheme="https://blog.malu.tech/tags/%E6%95%B0%E6%8D%AE%E5%88%86%E6%9E%90%E5%B9%B3%E5%8F%B0/"/>
    <content>
      <![CDATA[<blockquote><p><strong>文章来源</strong>：<a href="https://zhuanlan.zhihu.com/p/142490087">Python Django+SQL+Pandas+Pyecharts自建在线数据分析平台（一）</a><br><strong>作者</strong>：<a href="https://www.zhihu.com/people/chen-cheng-76-40">ccpic</a><br><strong>感谢</strong>：感谢作者 ccpic 分享的优质内容，本网页主要用于学习知识的存档备份，欢迎点击原网页支持作者。</p></blockquote><p>本篇是系列文章的第5篇，之前的更新见：</p><p>（一）需求分析&amp;技术实现</p><p>（二）初步搭建Django环境</p><p>（三）页面布局&amp;Django模板</p><p>（四）SQL+Pandas初步处理数据</p><h2 id="（五）前端表单交互"><a href="#（五）前端表单交互" class="headerlink" title="（五）前端表单交互"></a>（五）前端表单交互</h2><p>（六）Ajax异步传参与加载</p><p>（七）前端数据格式的处理</p><p>（八）DataTables接管前端表格</p><p>（九）Pyecharts实现交互图表</p><p>（十）静态图表的展示</p><p>（十一）“导出数据至Excel”功能</p><p>（十二）添加和配置缓存</p><p>（十三）用户登录系统</p><p>（十四）部署Django至生产环境</p><p>在上一章，我们已经成功从后端将分析结果传回前端Django模板并展示，但这个分析结果是静态的，缺乏交互性。本章我们希望在预留的filter.html模板内建立表单，从前端向后端提交数据筛选的参数。本章的内容比较容易理解，但对用户体验至关重要，是个细致活。</p><p>还是回到上一章对数据本身各字段的分析，这对表单设计也格外重要：<br><img src="/images/python-djangosqlpand/v2-059d3970defeaeb339abb7a9158d692f_1440w.webp"></p><p>第4章的纸上谈兵本章依然有用</p><p>如上图，我们需要有一个必填单选代表一个分析目标字段（我还是习惯称之为breakout字段），它决定返回的数据结果里是品类份额，还是品牌份额，还是其他xx份额。它也是后端Pandas的pivot_table方法里column的动态参数。</p><p>我们还需要另外两个必填单选字段——UNIT和PERIOD，原因也请参考上图。</p><p>而我们所有的属性字段都是可为空的多选。</p><p>本例中我们表单不需要考虑AMOUNT和DATE字段。因为AMOUNT是唯一的指标字段，而我们的分析结果会取最新一个DATE做横断面结果，并计划把所有DATE的数据作为趋势分析，我们不需要对DATE动态选择。而在其他一些场景下，日期字段是经常作为表单的一员的，甚至有很多专门为其设计的calendar控件。</p><p>综上所述，我们的表单设计是下面这个样子，我们需要在filter.html文件中实现它。<br><img src="/images/python-djangosqlpand/v2-aaa0687480ad010aa0204b71dbd28e57_1440w.webp"></p><p>TC为Therapy Class的简写，可理解为其他行业的不同层级的品类</p><p>实际前端模板代码编写前，可以后端先传一个预设的字段字典。这样操作一是分离前端方便以后修改，大部分情况下以后只修改后端就可以了；二是可以利用循环极大缩短代码长度，更加elegant。</p><p>我们再次修改views.py里index方法的代码，在context字典内增加表单的预设值传至前端：</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="comment"># 该字典key为前端准备显示的所有多选字段名, value为数据库对应的字段名</span></span><br><span class="line">D_MULTI_SELECT = &#123;</span><br><span class="line">    <span class="string">&#x27;TC I&#x27;</span>: <span class="string">&#x27;[TC I]&#x27;</span>,</span><br><span class="line">    <span class="string">&#x27;TC II&#x27;</span>: <span class="string">&#x27;[TC II]&#x27;</span>,</span><br><span class="line">    <span class="string">&#x27;TC III&#x27;</span>: <span class="string">&#x27;[TC III]&#x27;</span>,</span><br><span class="line">    <span class="string">&#x27;TC IV&#x27;</span>: <span class="string">&#x27;[TC IV]&#x27;</span>,</span><br><span class="line">    <span class="string">&#x27;通用名|MOLECULE&#x27;</span>: <span class="string">&#x27;MOLECULE&#x27;</span>,</span><br><span class="line">    <span class="string">&#x27;商品名|PRODUCT&#x27;</span>: <span class="string">&#x27;PRODUCT&#x27;</span>,</span><br><span class="line">    <span class="string">&#x27;包装|PACKAGE&#x27;</span>: <span class="string">&#x27;PACKAGE&#x27;</span>,</span><br><span class="line">    <span class="string">&#x27;生产企业|CORPORATION&#x27;</span>: <span class="string">&#x27;CORPORATION&#x27;</span>,</span><br><span class="line">    <span class="string">&#x27;企业类型&#x27;</span>: <span class="string">&#x27;MANUF_TYPE&#x27;</span>,</span><br><span class="line">    <span class="string">&#x27;剂型&#x27;</span>: <span class="string">&#x27;FORMULATION&#x27;</span>,</span><br><span class="line">    <span class="string">&#x27;剂量&#x27;</span>: <span class="string">&#x27;STRENGTH&#x27;</span></span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="keyword">def</span> <span class="title function_">index</span>(<span class="params">request</span>):</span><br><span class="line">    </span><br><span class="line">    ...</span><br><span class="line"></span><br><span class="line">    mselect_dict = &#123;&#125;</span><br><span class="line">    <span class="keyword">for</span> key, value <span class="keyword">in</span> D_MULTI_SELECT.items():</span><br><span class="line">        mselect_dict[key] = &#123;&#125;</span><br><span class="line">        mselect_dict[key][<span class="string">&#x27;select&#x27;</span>] = value</span><br><span class="line">        <span class="comment"># mselect_dict[key][&#x27;options&#x27;] = option_list 以后可以后端通过列表为每个多选控件传递备选项</span></span><br><span class="line">    </span><br><span class="line">    context = &#123;</span><br><span class="line">       ...</span><br><span class="line">       <span class="string">&#x27;mselect_dict&#x27;</span>: mselect_dict</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="keyword">return</span> render(request, <span class="string">&#x27;chpa_data/analysis.html&#x27;</span>, context) <span class="comment"># 注意本句和前一章也有变化，渲染至analysis.html而不是display.html</span></span><br></pre></td></tr></table></figure><p>前端html模板filter.html代码如下，为了用户体验，我们希望所有的下拉菜单都使用Semantic UI的search dropdown提供搜索响应功能，主要就是应用这个class：class&#x3D;”ui fluid search dropdown”：</p><figure class="highlight html"><table><tr><td class="code"><pre><span class="line"><span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;ui container&quot;</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;ui form&quot;</span>&gt;</span></span><br><span class="line">        <span class="tag">&lt;<span class="name">form</span> <span class="attr">action</span>=<span class="string">&quot;&quot;</span> <span class="attr">method</span>=<span class="string">&quot;post&quot;</span>&gt;</span></span><br><span class="line">            <span class="comment">&lt;!-- 在Django所有的 POST 表单元素时，需要加上下方的csrf_token tag，主要是安全方面的机制，本例后续使用AJAX方法，这里的POST class和token都不生效 --&gt;</span></span><br><span class="line">            &#123;<span class="symbol">&amp;#37;</span> csrf_token <span class="symbol">&amp;#37;</span>&#125;</span><br><span class="line">            <span class="tag">&lt;<span class="name">h3</span> <span class="attr">class</span>=<span class="string">&quot;ui header&quot;</span> <span class="attr">id</span>=<span class="string">&quot;analysis&quot;</span>&gt;</span>分析维度<span class="tag">&lt;/<span class="name">h3</span>&gt;</span></span><br><span class="line">            <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;field&quot;</span>&gt;</span></span><br><span class="line">                <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;fields&quot;</span>&gt;</span></span><br><span class="line">                    <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;sixteen wide field&quot;</span>&gt;</span></span><br><span class="line">                        <span class="tag">&lt;<span class="name">select</span> <span class="attr">name</span>=<span class="string">&quot;DIMENSION_select&quot;</span> <span class="attr">id</span>=<span class="string">&quot;DIMENSION_select&quot;</span> <span class="attr">class</span>=<span class="string">&quot;ui fluid search dropdown&quot;</span>&gt;</span></span><br><span class="line">                            &#123;<span class="symbol">&amp;#37;</span> for key, value in mselect_dict.items <span class="symbol">&amp;#37;</span>&#125;</span><br><span class="line">                                &#123;<span class="symbol">&amp;#37;</span> if value.select == &#x27;PRODUCT&#x27; <span class="symbol">&amp;#37;</span>&#125;</span><br><span class="line">                                    <span class="tag">&lt;<span class="name">option</span> <span class="attr">value</span>=<span class="string">&quot;&#123;<span class="symbol">&amp;#123;</span> value.select <span class="symbol">&amp;#125;</span>&#125;&quot;</span> <span class="attr">selected</span>&gt;</span>&#123;<span class="symbol">&amp;#123;</span> key <span class="symbol">&amp;#125;</span>&#125;<span class="tag">&lt;/<span class="name">option</span>&gt;</span></span><br><span class="line">                                &#123;<span class="symbol">&amp;#37;</span> else <span class="symbol">&amp;#37;</span>&#125;</span><br><span class="line">                                    <span class="tag">&lt;<span class="name">option</span> <span class="attr">value</span>=<span class="string">&quot;&#123;<span class="symbol">&amp;#123;</span> value.select <span class="symbol">&amp;#125;</span>&#125;&quot;</span>&gt;</span>&#123;<span class="symbol">&amp;#123;</span> key <span class="symbol">&amp;#125;</span>&#125;<span class="tag">&lt;/<span class="name">option</span>&gt;</span></span><br><span class="line">                                &#123;<span class="symbol">&amp;#37;</span> endif <span class="symbol">&amp;#37;</span>&#125;</span><br><span class="line">                            &#123;<span class="symbol">&amp;#37;</span> endfor <span class="symbol">&amp;#37;</span>&#125;</span><br><span class="line">                        <span class="tag">&lt;/<span class="name">select</span>&gt;</span></span><br><span class="line">                    <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">                <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">                <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;fields&quot;</span>&gt;</span></span><br><span class="line">                    <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;eight wide field&quot;</span>&gt;</span></span><br><span class="line">                        <span class="tag">&lt;<span class="name">select</span> <span class="attr">name</span>=<span class="string">&quot;UNIT_select&quot;</span> <span class="attr">id</span>=<span class="string">&quot;UNIT_select&quot;</span> <span class="attr">class</span>=<span class="string">&quot;ui fluid search dropdown&quot;</span>&gt;</span></span><br><span class="line">                            <span class="tag">&lt;<span class="name">option</span> <span class="attr">value</span>=<span class="string">&quot;Value&quot;</span> <span class="attr">selected</span>&gt;</span>金额<span class="tag">&lt;/<span class="name">option</span>&gt;</span></span><br><span class="line">                            <span class="tag">&lt;<span class="name">option</span> <span class="attr">value</span>=<span class="string">&quot;Volume&quot;</span>&gt;</span>盒数<span class="tag">&lt;/<span class="name">option</span>&gt;</span></span><br><span class="line">                            <span class="tag">&lt;<span class="name">option</span> <span class="attr">value</span>=<span class="string">&quot;Volume (Counting Unit)&quot;</span>&gt;</span>最小制剂单位数<span class="tag">&lt;/<span class="name">option</span>&gt;</span></span><br><span class="line">                        <span class="tag">&lt;/<span class="name">select</span>&gt;</span></span><br><span class="line">                    <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">                    <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;eight wide field&quot;</span>&gt;</span></span><br><span class="line">                        <span class="tag">&lt;<span class="name">select</span> <span class="attr">name</span>=<span class="string">&quot;PERIOD_select&quot;</span> <span class="attr">id</span>=<span class="string">&quot;PERIOD_select&quot;</span> <span class="attr">class</span>=<span class="string">&quot;ui fluid search dropdown&quot;</span>&gt;</span></span><br><span class="line">                            <span class="tag">&lt;<span class="name">option</span> <span class="attr">value</span>=<span class="string">&quot;MAT&quot;</span> <span class="attr">selected</span>&gt;</span>滚动年<span class="tag">&lt;/<span class="name">option</span>&gt;</span></span><br><span class="line">                            <span class="tag">&lt;<span class="name">option</span> <span class="attr">value</span>=<span class="string">&quot;QTR&quot;</span>&gt;</span>季度<span class="tag">&lt;/<span class="name">option</span>&gt;</span></span><br><span class="line">                        <span class="tag">&lt;/<span class="name">select</span>&gt;</span></span><br><span class="line">                    <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">                <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">            <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">            <span class="tag">&lt;<span class="name">h3</span> <span class="attr">class</span>=<span class="string">&quot;ui header&quot;</span> <span class="attr">id</span>=<span class="string">&quot;data_filter&quot;</span>&gt;</span>数据筛选<span class="tag">&lt;/<span class="name">h3</span>&gt;</span></span><br><span class="line">            <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;field&quot;</span>&gt;</span></span><br><span class="line">                &#123;<span class="symbol">&amp;#37;</span> for key, value in mselect_dict.items <span class="symbol">&amp;#37;</span>&#125;</span><br><span class="line">                <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;field&quot;</span>&gt;</span></span><br><span class="line">                    <span class="tag">&lt;<span class="name">select</span> <span class="attr">name</span>=<span class="string">&quot;&#123;<span class="symbol">&amp;#123;</span> value.select|add:&quot;</span><span class="attr">_select</span>[]&quot; &amp;#<span class="attr">125</span>;&#125;&quot; <span class="attr">id</span>=<span class="string">&quot;&#123;<span class="symbol">&amp;#123;</span> value.select|add:&quot;</span><span class="attr">_select</span>&quot; &amp;#<span class="attr">125</span>;&#125;&quot; <span class="attr">multiple</span>=<span class="string">&quot;&quot;</span></span></span><br><span class="line"><span class="tag">                            <span class="attr">class</span>=<span class="string">&quot;ui fluid search dropdown&quot;</span>&gt;</span></span><br><span class="line">                        <span class="tag">&lt;<span class="name">option</span> <span class="attr">value</span>=<span class="string">&quot;&quot;</span>&gt;</span>&#123;<span class="symbol">&amp;#123;</span> key <span class="symbol">&amp;#125;</span>&#125;<span class="tag">&lt;/<span class="name">option</span>&gt;</span></span><br><span class="line">&#123;#                        &#123;<span class="symbol">&amp;#37;</span> for item in value.options <span class="symbol">&amp;#37;</span>&#125;#&#125;</span><br><span class="line">&#123;#                            <span class="tag">&lt;<span class="name">option</span> <span class="attr">value</span>=<span class="string">&quot;&#123;<span class="symbol">&amp;#123;</span> item <span class="symbol">&amp;#125;</span>&#125;&quot;</span>&gt;</span>&#123;<span class="symbol">&amp;#123;</span> item <span class="symbol">&amp;#125;</span>&#125;<span class="tag">&lt;/<span class="name">option</span>&gt;</span>#&#125;</span><br><span class="line">&#123;#                        &#123;<span class="symbol">&amp;#37;</span> endfor <span class="symbol">&amp;#37;</span>&#125;#&#125;</span><br><span class="line">                    <span class="tag">&lt;/<span class="name">select</span>&gt;</span></span><br><span class="line">                <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">                &#123;<span class="symbol">&amp;#37;</span> endfor <span class="symbol">&amp;#37;</span>&#125;</span><br><span class="line">            <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">            <span class="tag">&lt;<span class="name">br</span>&gt;</span></span><br><span class="line">            <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;ui buttons&quot;</span>&gt;</span></span><br><span class="line">                <span class="tag">&lt;<span class="name">input</span> <span class="attr">class</span>=<span class="string">&quot;ui blue button&quot;</span> <span class="attr">type</span>=<span class="string">&#x27;button&#x27;</span> <span class="attr">id</span>=<span class="string">&#x27;AJAX_get&#x27;</span> <span class="attr">value</span>=<span class="string">&quot;查询&quot;</span>/&gt;</span></span><br><span class="line">            <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">        <span class="tag">&lt;/<span class="name">form</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line"><span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line"></span><br><span class="line"><span class="comment">&lt;!-- 因为用到Semantic UI的Search Dropdown控件，必须有下面语句初始化 --&gt;</span></span><br><span class="line"><span class="tag">&lt;<span class="name">script</span>&gt;</span><span class="language-javascript"></span></span><br><span class="line"><span class="language-javascript">    $(<span class="string">&#x27;.ui.fluid.search.dropdown&#x27;</span>)</span></span><br><span class="line"><span class="language-javascript">        .<span class="title function_">dropdown</span>(&#123; <span class="attr">fullTextSearch</span>: <span class="literal">true</span> &#125;);</span></span><br><span class="line"><span class="language-javascript"></span><span class="tag">&lt;/<span class="name">script</span>&gt;</span></span><br></pre></td></tr></table></figure><p>这里首先我们第一次遇到了Django&#x2F;Jinja2模板语法的集中应用，因为本文没有使用Django ORM，这种应用后续出场不多。我们只需要明白{&#37; &#37;}是功能标签，而{&#123; &#125;}是变量标签，类似在模板层面的简单编程。而下方代码的意思是循环遍历后方传来的mselect_dict字典，字典的key是单选dimension_select下拉菜单选项的text，而value里嵌套的select键的值是菜单选项的value：</p><figure class="highlight html"><table><tr><td class="code"><pre><span class="line"><span class="tag">&lt;<span class="name">select</span> <span class="attr">name</span>=<span class="string">&quot;DIMENSION_select&quot;</span> <span class="attr">id</span>=<span class="string">&quot;DIMENSION_select&quot;</span> <span class="attr">class</span>=<span class="string">&quot;ui fluid search dropdown&quot;</span>&gt;</span></span><br><span class="line">    &#123;<span class="symbol">&amp;#37;</span> for key, value in mselect_dict.items <span class="symbol">&amp;#37;</span>&#125;</span><br><span class="line">        &#123;<span class="symbol">&amp;#37;</span> if value.select == &#x27;PRODUCT&#x27; <span class="symbol">&amp;#37;</span>&#125;</span><br><span class="line">            <span class="tag">&lt;<span class="name">option</span> <span class="attr">value</span>=<span class="string">&quot;&#123;<span class="symbol">&amp;#123;</span> value.select <span class="symbol">&amp;#125;</span>&#125;&quot;</span> <span class="attr">selected</span>&gt;</span>&#123;<span class="symbol">&amp;#123;</span> key <span class="symbol">&amp;#125;</span>&#125;<span class="tag">&lt;/<span class="name">option</span>&gt;</span></span><br><span class="line">        &#123;<span class="symbol">&amp;#37;</span> else <span class="symbol">&amp;#37;</span>&#125;</span><br><span class="line">            <span class="tag">&lt;<span class="name">option</span> <span class="attr">value</span>=<span class="string">&quot;&#123;<span class="symbol">&amp;#123;</span> value.select <span class="symbol">&amp;#125;</span>&#125;&quot;</span>&gt;</span>&#123;<span class="symbol">&amp;#123;</span> key <span class="symbol">&amp;#125;</span>&#125;<span class="tag">&lt;/<span class="name">option</span>&gt;</span></span><br><span class="line">        &#123;<span class="symbol">&amp;#37;</span> endif <span class="symbol">&amp;#37;</span>&#125;</span><br><span class="line">    &#123;<span class="symbol">&amp;#37;</span> endfor <span class="symbol">&amp;#37;</span>&#125;</span><br><span class="line"><span class="tag">&lt;/<span class="name">select</span>&gt;</span></span><br></pre></td></tr></table></figure><p>同理，后续又循环了一次mselect_dict，为根据字典内容生成若干个多选下拉菜单，注释掉的部分是后端动态生成备选项的一种解决方案，本文后半部分会涉及：</p><figure class="highlight html"><table><tr><td class="code"><pre><span class="line">&#123;<span class="symbol">&amp;#37;</span> for key, value in mselect_dict.items <span class="symbol">&amp;#37;</span>&#125;</span><br><span class="line">    <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;field&quot;</span>&gt;</span></span><br><span class="line">        <span class="tag">&lt;<span class="name">select</span> <span class="attr">name</span>=<span class="string">&quot;&#123;<span class="symbol">&amp;#123;</span> value.select|add:&quot;</span><span class="attr">_select</span>[]&quot; &amp;#<span class="attr">125</span>;&#125;&quot; <span class="attr">id</span>=<span class="string">&quot;&#123;<span class="symbol">&amp;#123;</span> value.select|add:&quot;</span><span class="attr">_select</span>&quot; &amp;#<span class="attr">125</span>;&#125;&quot;</span></span><br><span class="line"><span class="tag">                <span class="attr">multiple</span>=<span class="string">&quot;&quot;</span></span></span><br><span class="line"><span class="tag">                <span class="attr">class</span>=<span class="string">&quot;ui fluid search dropdown&quot;</span>&gt;</span></span><br><span class="line">            <span class="tag">&lt;<span class="name">option</span> <span class="attr">value</span>=<span class="string">&quot;&quot;</span>&gt;</span>&#123;<span class="symbol">&amp;#123;</span> key <span class="symbol">&amp;#125;</span>&#125;<span class="tag">&lt;/<span class="name">option</span>&gt;</span></span><br><span class="line">&#123;#            &#123;<span class="symbol">&amp;#37;</span> for item in value.options <span class="symbol">&amp;#37;</span>&#125;#&#125;</span><br><span class="line">&#123;#                <span class="tag">&lt;<span class="name">option</span> <span class="attr">value</span>=<span class="string">&quot;&#123;<span class="symbol">&amp;#123;</span> item <span class="symbol">&amp;#125;</span>&#125;&quot;</span>&gt;</span>&#123;<span class="symbol">&amp;#123;</span> item <span class="symbol">&amp;#125;</span>&#125;<span class="tag">&lt;/<span class="name">option</span>&gt;</span>#&#125;</span><br><span class="line">&#123;#            &#123;<span class="symbol">&amp;#37;</span> endfor <span class="symbol">&amp;#37;</span>&#125;#&#125;</span><br><span class="line">        <span class="tag">&lt;/<span class="name">select</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">&#123;<span class="symbol">&amp;#37;</span> endfor <span class="symbol">&amp;#37;</span>&#125;</span><br></pre></td></tr></table></figure><p>这里有一个大坑是下面这句，可能会让人觉得很奇怪（这里的|add是tag filter，下一章会解释，这并不是最奇怪的地方）：</p><figure class="highlight html"><table><tr><td class="code"><pre><span class="line"><span class="tag">&lt;<span class="name">select</span> <span class="attr">name</span>=<span class="string">&quot;&#123;<span class="symbol">&amp;#123;</span> value.select|add:&quot;</span><span class="attr">_select</span>[]&quot; &amp;#<span class="attr">125</span>;&#125;&quot; <span class="attr">id</span>=<span class="string">&quot;&#123;<span class="symbol">&amp;#123;</span> value.select|add:&quot;</span><span class="attr">_select</span>&quot; &amp;#<span class="attr">125</span>;&#125;&quot; <span class="attr">multiple</span>=<span class="string">&quot;&quot;</span></span></span><br><span class="line"><span class="tag">                            <span class="attr">class</span>=<span class="string">&quot;ui fluid search dropdown&quot;</span>&gt;</span></span><br></pre></td></tr></table></figure><p>为什么<select name>要加个后缀[]？这是因为以后在jQuery传参时<strong>多选</strong>控件（实际就是传送array而不是单个变量的控件）的<select name>在很多场景下（如后续章节介绍的AJAX异步传参）必须以[]结尾才能正确工作。但有时[]也不是必须的。</p><p>此时再访问我们的主页<a href="http://127.0.0.1:8088/chpa/index%EF%BC%8C%E7%95%8C%E9%9D%A2%E5%B7%B2%E7%BB%8F%E5%8F%98%E6%88%90%E4%BA%86%E4%B8%8B%E9%9D%A2%E8%BF%99%E6%A0%B7%EF%BC%9A">http://127.0.0.1:8088/chpa/index，界面已经变成了下面这样：</a></p><p>筛选框已经在那了，但下方的多选框点开还没选项，我们还需要一个步骤，从后端动态传入所有多选下拉菜单的备选选项。<br><img src="/images/python-djangosqlpand/v2-445483153498eab3c25a6e3e50801dbe_1440w.webp"></p><p>此时有两种常用方法：</p><ul><li><strong>在页面初始化时从后端提取所有字段的不重复值作为选项传入前端。</strong></li><li><strong>在控件搜索时根据键入关键字实时从后端返回前n个相关备选项。</strong></li></ul><p>第一种方法的优点是简单直接。在上方的代码块中，我们其实已经预留了注释掉的相应的代码，将views.py的index方法修改成类似下面这样，增加option_list部分传至前端：</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">def</span> <span class="title function_">index</span>(<span class="params">request</span>):</span><br><span class="line">    </span><br><span class="line">    ...</span><br><span class="line"></span><br><span class="line">    mselect_dict = &#123;&#125;</span><br><span class="line">    <span class="keyword">for</span> key, value <span class="keyword">in</span> D_FIELD.items():</span><br><span class="line">        mselect_dict[key] = &#123;&#125;</span><br><span class="line">        mselect_dict[key][<span class="string">&#x27;select&#x27;</span>] = value</span><br><span class="line">        mselect_dict[key][<span class="string">&#x27;options&#x27;</span>] = option_list <span class="comment"># option_list可以通过sql Distinct语句或Pandas的Unique方法获得，在此不再赘述</span></span><br><span class="line">    </span><br><span class="line">    ...</span><br><span class="line"></span><br><span class="line"><span class="comment"># 下面是一个获得各个字段option_list的简单方法</span></span><br><span class="line"><span class="keyword">def</span> <span class="title function_">get_distinct_list</span>(<span class="params">column, db_table</span>):</span><br><span class="line">    sql = <span class="string">&quot;Select DISTINCT &quot;</span> + column + <span class="string">&quot; From &quot;</span> + db_table</span><br><span class="line">    df = pd.read_sql_query(sql, ENGINE)</span><br><span class="line">    l = df.values.flatten().tolist()</span><br><span class="line">    <span class="keyword">return</span> l</span><br></pre></td></tr></table></figure><p>再在前端filter.html用下面的循环语句渲染<option></option>部分：</p><figure class="highlight html"><table><tr><td class="code"><pre><span class="line"><span class="tag">&lt;<span class="name">h3</span> <span class="attr">class</span>=<span class="string">&quot;ui header&quot;</span> <span class="attr">id</span>=<span class="string">&quot;data_filter&quot;</span>&gt;</span>数据筛选<span class="tag">&lt;/<span class="name">h3</span>&gt;</span></span><br><span class="line"><span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;field&quot;</span>&gt;</span></span><br><span class="line">    &#123;<span class="symbol">&amp;#37;</span> for key, value in mselect_dict.items <span class="symbol">&amp;#37;</span>&#125;</span><br><span class="line">    <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;field&quot;</span>&gt;</span></span><br><span class="line">        <span class="tag">&lt;<span class="name">select</span> <span class="attr">name</span>=<span class="string">&quot;&#123;<span class="symbol">&amp;#123;</span> value.select|add:&quot;</span><span class="attr">_select</span>[]&quot; &amp;#<span class="attr">125</span>;&#125;&quot; <span class="attr">id</span>=<span class="string">&quot;&#123;<span class="symbol">&amp;#123;</span> value.select|add:&quot;</span><span class="attr">_select</span>&quot; &amp;#<span class="attr">125</span>;&#125;&quot; <span class="attr">multiple</span>=<span class="string">&quot;&quot;</span></span></span><br><span class="line"><span class="tag">                <span class="attr">class</span>=<span class="string">&quot;ui fluid search dropdown&quot;</span>&gt;</span></span><br><span class="line">            <span class="tag">&lt;<span class="name">option</span> <span class="attr">value</span>=<span class="string">&quot;&quot;</span>&gt;</span>&#123;<span class="symbol">&amp;#123;</span> key <span class="symbol">&amp;#125;</span>&#125;<span class="tag">&lt;/<span class="name">option</span>&gt;</span></span><br><span class="line">            &#123;<span class="symbol">&amp;#37;</span> for item in value.options <span class="symbol">&amp;#37;</span>&#125;</span><br><span class="line">                <span class="tag">&lt;<span class="name">option</span> <span class="attr">value</span>=<span class="string">&quot;&#123;<span class="symbol">&amp;#123;</span> item <span class="symbol">&amp;#125;</span>&#125;&quot;</span>&gt;</span>&#123;<span class="symbol">&amp;#123;</span> item <span class="symbol">&amp;#125;</span>&#125;<span class="tag">&lt;/<span class="name">option</span>&gt;</span></span><br><span class="line">            &#123;<span class="symbol">&amp;#37;</span> endfor <span class="symbol">&amp;#37;</span>&#125;</span><br><span class="line">        <span class="tag">&lt;/<span class="name">select</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">    &#123;<span class="symbol">&amp;#37;</span> endfor <span class="symbol">&amp;#37;</span>&#125;</span><br><span class="line"><span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br></pre></td></tr></table></figure><p>很遗憾，功能是实现了，但用户体验很不好。因为我们部分字段的可选项过多，造成页面初始化加载很慢，并且点开选项较多的下拉菜单时反应也很慢。这也是初始化控件选项方法的最大缺点，不适应加载量太大的情况。<br><img src="/images/python-djangosqlpand/v2-fd7ba384d58d993e9681da14cba0ab77_1440w.webp"></p><p>下拉菜单的Search Select功能实现了，但加载时间不可接受</p><p>但是我们必须使用search select功能，因为医药行业的专业术语太多了。于是考虑使用第二个方法，在控件搜索时根据键入关键字实时从后端返回前n个相关备选项，也就是我们说的on Server Response的方法。该方法适合表单可选项过多的场景。不使用Vue或React的情况下，Semantic UI的dropdown API就支持建设这种响应式搜索功能，并且官网提供了下方的例子：</p><p>本例中实现这种方法确实要相对复杂。我们需要先在views.py建立search方法，该方法除request外包含2个参数，要查询的字段名和查询的字符串，返回不重复的匹配结果作为前端表单选项，格式为符合Semantic UI要求格式的json。</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">import</span> json</span><br><span class="line"></span><br><span class="line"><span class="keyword">def</span> <span class="title function_">search</span>(<span class="params">request, column, kw</span>):</span><br><span class="line">    sql = <span class="string">&quot;SELECT DISTINCT TOP 10 %s FROM %s WHERE %s like &#x27;%%%s%%&#x27;&quot;</span> % (column, DB_TABLE, column, kw) <span class="comment"># 最简单的单一字符串like，返回不重复的前10个结果</span></span><br><span class="line">    <span class="keyword">try</span>:</span><br><span class="line">        df = pd.read_sql_query(sql, ENGINE)</span><br><span class="line">        l = df.values.flatten().tolist()</span><br><span class="line">        results_list = []</span><br><span class="line">        <span class="keyword">for</span> element <span class="keyword">in</span> l:</span><br><span class="line">            option_dict = &#123;<span class="string">&#x27;name&#x27;</span>: element,</span><br><span class="line">                           <span class="string">&#x27;value&#x27;</span>: element,</span><br><span class="line">                           &#125;</span><br><span class="line">            results_list.append(option_dict)</span><br><span class="line">        res = &#123;</span><br><span class="line">            <span class="string">&quot;success&quot;</span>: <span class="literal">True</span>,</span><br><span class="line">            <span class="string">&quot;results&quot;</span>: results_list,</span><br><span class="line">            <span class="string">&quot;code&quot;</span>: <span class="number">200</span>,</span><br><span class="line">        &#125;</span><br><span class="line">    <span class="keyword">except</span> Exception <span class="keyword">as</span> e:</span><br><span class="line">        res = &#123;</span><br><span class="line">            <span class="string">&quot;success&quot;</span>: <span class="literal">False</span>,</span><br><span class="line">            <span class="string">&quot;errMsg&quot;</span>: e,</span><br><span class="line">            <span class="string">&quot;code&quot;</span>: <span class="number">0</span>,</span><br><span class="line">        &#125;</span><br><span class="line">    <span class="keyword">return</span> HttpResponse(json.dumps(res, ensure_ascii=<span class="literal">False</span>), content_type=<span class="string">&quot;application/json charset=utf-8&quot;</span>) <span class="comment"># 返回结果必须是json格式</span></span><br></pre></td></tr></table></figure><p>上面只是个匹配关键字的最简单例子，未来还可以继续完善，例如处理多个关键字，模糊查询等。</p><p>同时，我们需要在url.py编辑对应search方法的URL pattern，并用&lt;&gt;括号预留column和kw两个对应的参数位置：</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line">urlpatterns = [</span><br><span class="line">    ...</span><br><span class="line">    path(<span class="string">r&#x27;search/&lt;str:column&gt;/&lt;str:kw&gt;&#x27;</span>, views.search, name=<span class="string">&#x27;search&#x27;</span>)</span><br><span class="line">]</span><br></pre></td></tr></table></figure><p>此时可在浏览器输入上面的URL试试看效果，能看到已经正常返回预期的json了：<br><img src="/images/python-djangosqlpand/v2-b9a12f78ebfd5fa4a5c9d8f2d16fa31c_1440w.webp"></p><p>最后参考Semantic UI官网的例子在前端模板文件filter.html末尾加上下面这段JS代码，将后台search方法和多选框绑定。注意下方代码相对复杂有好几个坑，我都在注释一一标出了：</p><figure class="highlight js"><table><tr><td class="code"><pre><span class="line">&lt;script&gt;</span><br><span class="line">    <span class="comment">// 在JS中再次使用字段字典，要加|safe不转义</span></span><br><span class="line">    <span class="keyword">var</span> dict = &#123;&amp;#<span class="number">123</span>; mselect_dict|safe &amp;#<span class="number">125</span>;&#125;;</span><br><span class="line">    <span class="comment">// 还是转义问题，在Django模板中遇到带有&#123;&#125;的html代码必须使用replace这种方式处理</span></span><br><span class="line">    <span class="keyword">var</span> url = <span class="string">&quot;&#123;&amp;#37; url &#x27;chpa:search&#x27; &#x27;COLUMNPLACEHOLDER&#x27; &#x27;QUERYPLACEHOLDER&#x27; &amp;#37;&#125;&quot;</span>.<span class="title function_">replace</span>(</span><br><span class="line">        <span class="string">&#x27;QUERYPLACEHOLDER&#x27;</span>, <span class="string">&#x27;&#123;query&#125;&#x27;</span></span><br><span class="line">    );</span><br><span class="line">    <span class="comment">// jQuery语法遍历所有多选框</span></span><br><span class="line">    $(<span class="string">&#x27;.ui.fluid.search.dropdown.selection.multiple&#x27;</span>).<span class="title function_">each</span>(<span class="keyword">function</span> (<span class="params"></span>) &#123;</span><br><span class="line">        <span class="comment">// Semantic UI语法获得多选框默认文本</span></span><br><span class="line">        <span class="keyword">var</span> text = $(<span class="variable language_">this</span>).<span class="title function_">dropdown</span>(<span class="string">&#x27;get default text&#x27;</span>);</span><br><span class="line">        <span class="comment">// 根据字典倒推该多选框是哪个字段</span></span><br><span class="line">        <span class="keyword">var</span> column = dict[text][<span class="string">&#x27;select&#x27;</span>];</span><br><span class="line">        $(<span class="variable language_">this</span>).<span class="title function_">dropdown</span>(</span><br><span class="line">            &#123;</span><br><span class="line">                <span class="attr">apiSettings</span>: &#123;</span><br><span class="line">                    <span class="comment">// 用下方URL从后端返回查询后的json</span></span><br><span class="line">                    <span class="attr">url</span>: url.<span class="title function_">replace</span>(<span class="string">&#x27;COLUMNPLACEHOLDER&#x27;</span>, column)</span><br><span class="line">                &#125;,</span><br><span class="line">                <span class="comment">// 输入至少2个字符后才query</span></span><br><span class="line">                minCharacters : <span class="number">2</span></span><br><span class="line">            &#125;)</span><br><span class="line">        ;</span><br><span class="line">    &#125;)</span><br><span class="line">&lt;/script&gt;</span><br></pre></td></tr></table></figure><p>在评论区有人回复下面语句会出现bug：</p><figure class="highlight js"><table><tr><td class="code"><pre><span class="line"><span class="comment">// Semantic UI语法获得多选框默认文本</span></span><br><span class="line"><span class="keyword">var</span> text = $(<span class="variable language_">this</span>).<span class="title function_">dropdown</span>(<span class="string">&#x27;get default text&#x27;</span>);</span><br></pre></td></tr></table></figure><p>虽然我个人没有碰到，但是如果有碰到的，可以考虑摒弃Semantic UI API，使用原生的JQuery语句：</p><figure class="highlight js"><table><tr><td class="code"><pre><span class="line"><span class="keyword">var</span> text = $(<span class="variable language_">this</span>).<span class="title function_">children</span>(<span class="string">&#x27;select&#x27;</span>).<span class="title function_">children</span>(<span class="string">&#x27;option:first&#x27;</span>).<span class="title function_">text</span>();</span><br></pre></td></tr></table></figure><p>至此，我们终于完成了大部分前端表单交互的表面工作。本章内容比较繁杂，又第一次在项目中引入了二手程序员的天敌JS，我们在此停笔告一段落。下一章再讨论传参和异步加载的话题。</p>]]>
    </content>
    <id>https://blog.malu.tech/python-django-sql-pandas-pyecharts-data-analysis-platform-5/</id>
    <link href="https://blog.malu.tech/python-django-sql-pandas-pyecharts-data-analysis-platform-5/"/>
    <published>2024-01-28T16:00:00.000Z</published>
    <summary>Python Django+SQL+Pandas+Pyecharts自建在线数据分析平台系列文章第五篇：前端表单交互</summary>
    <title>Python Django+SQL+Pandas+Pyecharts自建在线数据分析平台（五）</title>
    <updated>2024-01-28T16:00:00.000Z</updated>
  </entry>
  <entry>
    <author>
      <name>Road</name>
    </author>
    <category term="Python" scheme="https://blog.malu.tech/categories/Python/"/>
    <category term="Django" scheme="https://blog.malu.tech/categories/Python/Django/"/>
    <category term="数据分析" scheme="https://blog.malu.tech/categories/Python/Django/%E6%95%B0%E6%8D%AE%E5%88%86%E6%9E%90/"/>
    <category term="Python" scheme="https://blog.malu.tech/tags/Python/"/>
    <category term="Django" scheme="https://blog.malu.tech/tags/Django/"/>
    <category term="SQL" scheme="https://blog.malu.tech/tags/SQL/"/>
    <category term="Pandas" scheme="https://blog.malu.tech/tags/Pandas/"/>
    <category term="Pyecharts" scheme="https://blog.malu.tech/tags/Pyecharts/"/>
    <category term="数据分析平台" scheme="https://blog.malu.tech/tags/%E6%95%B0%E6%8D%AE%E5%88%86%E6%9E%90%E5%B9%B3%E5%8F%B0/"/>
    <content>
      <![CDATA[<blockquote><p><strong>文章来源</strong>：<a href="https://zhuanlan.zhihu.com/p/142490087">Python Django+SQL+Pandas+Pyecharts自建在线数据分析平台（一）</a><br><strong>作者</strong>：<a href="https://www.zhihu.com/people/chen-cheng-76-40">ccpic</a><br><strong>感谢</strong>：感谢作者 ccpic 分享的优质内容，本网页主要用于学习知识的存档备份，欢迎点击原网页支持作者。</p></blockquote><p>本篇是系列文章的第4篇：</p><p>（一）需求分析&amp;技术实现</p><p>（二）初步搭建Django环境</p><p>（三）页面布局&amp;Django模板</p><p>（五）前端表单交互</p><p>（六）Ajax异步传参与加载</p><p>（七）前端数据格式的处理</p><p>（八）DataTables接管前端表格</p><p>（九）Pyecharts实现交互图表</p><p>（十）静态图表的展示</p><p>（十一）“导出数据至Excel”功能</p><p>（十二）添加和配置缓存</p><p>（十三）用户登录系统</p><p>（十四）部署Django至生产环境</p><p>在前两章关注架构的搭建后，把目光拉回到数据的处理这边。我们希望能继续贯彻第一、二章的核心思路，使用SqlAlchemy从服务器调取数据，然后Pandas将数据处理成我们想要的结果，再渲染到第三章创建的display.html模板去。不同于第二章末尾的简单例子，这次我们会着重于Pandas处理数据的部分，争取呈现出一些更加丰富有价值的结果。</p><p>涉及数据就必然与业务牵扯上关系，行业与行业之间是差别极大的。这里只能把实际情况写下来，作举一反三之用。</p><p>Dummy测试数据链接：<a href="https://share.weiyun.com/q1EZl8lW">https://share.weiyun.com/q1EZl8lW</a> 密码：xaw3kg</p><p>在动手写程序之前，还是应当把我们的数据字段分析透彻，这是一份药物销售数据：<br><img src="/images/python-djangosqlpand/v2-e474c59daa902e1d4f1ce4ed18c05867_1440w.webp"></p><p>观察之后，我们可以把这些字段分为四类。</p><ul><li><strong>属性字段</strong> - 前13个字段是描述这条记录中主体药品的属性，包括分类，化合物通用名，剂型，生产公司等。</li><li><strong>指标字段</strong> - AMOUNT是每行数据的唯一量化指标字段</li><li><strong>筛选字段</strong> - UNIT，PERIOD字段则是描述量化指标范围的。UNIT区分该指标是销售额，销售量（盒数）或者销售量（片数）；PERIOD区别该指标是滚动年还是季度。</li><li><strong>日期字段</strong> - DATE，很好理解无需解释了。</li></ul><p>这么分类对于厘清分析思路是很重要的，我们尝试用同样的数据手动拉个Excel数据透视表加深我们的理解：<br><img src="/images/python-djangosqlpand/v2-02a502f31e033612d4173b75ec2ac034_1440w.webp"></p><p>仔细品味以上数据透视表右下角的设置和左边结果，我们可以得出：</p><ul><li><strong>属性字段</strong> - 其中有至少一个属性字段是作为我们分析的breakout的（如品类竞争，品牌竞争，剂型分布），这个字段适合放在数据透视表的列或行。多个分析目标可以嵌套。其他的非分析目标的字段可以酌情作为筛选。</li><li><strong>指标字段</strong> - AMOUNT是我们汇总统计的量化对象，这个字段是永远需要放在数据透视表的值那里，汇总方式大部分时候是求和。</li><li><strong>筛选字段</strong> -UNIT和PERIOD是永远需要优先筛选的，且一定是单选。因为把销售额和销售量相加的量化指标没有任何意义，把滚动年和季度相加的时间段也不符合逻辑。所以这两个字段需要永远存在于数据透视表的筛选器，且有一个已选项。<br><img src="/images/python-djangosqlpand/v2-00249316401d2499c8b379cb70b24c9d_1440w.webp"></li><li><strong>日期字段</strong> - DATE很微妙，它在一些横截面数据分析时（如最新xx，同比增长）是个筛选字段：</li></ul><p><img src="/images/python-djangosqlpand/v2-00249316401d2499c8b379cb70b24c9d_1440w.webp"></p><p>DATA在横截面数据分析时是筛选字段</p><p>但她在趋势分析时又经常作为数据透视的行或列出场，作为结果的一个维度。<br><img src="/images/python-djangosqlpand/v2-eaec7f9e95f4e9361006a3c1fc905682_1440w.webp"></p><p>DATE在数据分析时经常以数据透视的行&#x2F;列（面板数据x轴）为角色出场</p><p>DATE还有一个特殊地方是它的筛选特别是多选遇到移动平均数据时要非常小心，必须在一定规则下进行筛选，见下方的示意图。<br><img src="/images/python-djangosqlpand/v2-d1d38d066b13057c7d9a1a0c4ef6c90a_1440w.webp"></p><p>我认为做以上如此多的纸上谈兵的工作是有价值的，能快速帮助我们明确以下一些问题：</p><ul><li><strong>明确如何编写从服务器提取数据的Sql语句</strong></li><li><strong>明确Pandas处理数据的一些细节，如使用pivot_table方法时value, index, column, aggfunc等参数分别是什么</strong></li><li><strong>明确前端未来数据筛选时的表单属性，如单选和多选的区别，必填和可选的区别</strong></li></ul><p>下面开始实际操作。<strong>我的思路还是用SQL尽可能精确地筛选数据，再由Pandas根据需求把数据做各种处理。个人觉得抛开运算速度，Pandas在应对数据处理的各种实际操作时更加友善和高效。</strong></p><p>根据上方的分析，我们有两个字段是必须提前做筛选的，PERIOD字段和UNIT字段，其他筛选字段则不确定。因此我们的取数SQL可以暂时写成这样：</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">def</span> <span class="title function_">sqlparse</span>(<span class="params">period, unit, filter_sql=<span class="literal">None</span></span>):</span><br><span class="line">    sql = <span class="string">&quot;Select * from %s Where PERIOD = &#x27;%s&#x27; And UNIT = &#x27;%s&#x27;&quot;</span> % (DB_TABLE, period, unit) <span class="comment"># 必选的两个筛选字段</span></span><br><span class="line">![](images/python-djangosqlpand/v2-a18c620ac5420379abc0d461d60b7fe1_1440w.webp)</span><br><span class="line">    <span class="keyword">if</span> filter_sql <span class="keyword">is</span> <span class="keyword">not</span> <span class="literal">None</span>:</span><br><span class="line">        sql = <span class="string">&quot;%s And %s&quot;</span> % (sql, filter_sql) <span class="comment"># 其他可选的筛选字段，如有则以And连接自定义字符串</span></span><br><span class="line">    <span class="keyword">return</span> sql</span><br></pre></td></tr></table></figure><p>未来跟前端结合后更复杂的SQL语句拼接会写成下面这个趋势</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">def</span> <span class="title function_">sqlparse</span>(<span class="params">context</span>):</span><br><span class="line">    sql = <span class="string">&quot;Select * from %s Where PERIOD = &#x27;%s&#x27; And UNIT = &#x27;%s&#x27;&quot;</span> % \</span><br><span class="line">          (DB_TABLE, context[<span class="string">&#x27;period_selected&#x27;</span>], context[<span class="string">&#x27;unit_selected&#x27;</span>]) <span class="comment"># context为前端通过表单传来的字典</span></span><br><span class="line">    <span class="comment"># sql = sql_extent(sql, &#x27;[TC I]&#x27;, context[&#x27;tc_i_selected&#x27;]) #未来可以通过进一步拼接字符串动态扩展sql语句</span></span><br><span class="line">    <span class="keyword">return</span> sql</span><br></pre></td></tr></table></figure><p>假设我们关心一类药ARB，它是治疗高血压的主流药物血管紧张素Ⅱ受体阻滞剂，我们关心这个市场滚动年的销售额，调用上方的sqlparse方法则可生成需要的sql语句，并依然使用Pandas的read_sql_query语句读取：</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line">sql = sqlparse(<span class="string">&#x27;MAT&#x27;</span>, <span class="string">&#x27;Value&#x27;</span>, <span class="string">&quot; [TC III] = &#x27;C09C ANGIOTENS-II ANTAG, PLAIN|血管紧张素II拮抗剂，单一用药&#x27;&quot;</span>) <span class="comment">#读取ARB市场的滚动年销售额数据</span></span><br><span class="line"><span class="comment">#字符串拼接后sql应为Select * from data Where PERIOD = &#x27;MAT&#x27; And UNIT = &#x27;Value&#x27; And  [TC III] = &#x27;C09C ANGIOTENS-II ANTAG, PLAIN|血管紧张素II拮抗剂，单一用药&#x27;</span></span><br><span class="line">df = pd.read_sql_query(sql, ENGINE)  <span class="comment"># 将sql语句结果读取至Pandas Dataframe</span></span><br></pre></td></tr></table></figure><p>此时只是对原始数做第一步筛选，数据结构并没有变，我们使用Pandas的pivoted_table方法快速透视数据获取结果。还记得文章之前对各个字段的分析，这里要据此为每个参数选择合适的字段：</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">import</span> numpy <span class="keyword">as</span> np</span><br><span class="line"></span><br><span class="line">...</span><br><span class="line"></span><br><span class="line">pivoted = pd.pivot_table(df,</span><br><span class="line">                       values=<span class="string">&#x27;AMOUNT&#x27;</span>,  <span class="comment"># 数据透视汇总值为AMOUNT字段，一般保持不变</span></span><br><span class="line">                       index=<span class="string">&#x27;DATE&#x27;</span>,  <span class="comment"># 数据透视行为DATE字段，一般保持不变</span></span><br><span class="line">                       columns=<span class="string">&#x27;MOLECULE&#x27;</span>,  <span class="comment"># 数据透视列为MOLECULE字段，该字段以后应跟随分析需要动态传参</span></span><br><span class="line">                       aggfunc=np.<span class="built_in">sum</span>) <span class="comment"># 数据透视汇总方式为求和，一般保持不变</span></span><br></pre></td></tr></table></figure><p>此时的数据被透视为下方的时间序列格式，这个结果某些输出已经可以直接使用了，做进一步其他数据处理也异常方便：<br><img src="/images/python-djangosqlpand/v2-3fcb953a11d6e3277ccacf96c952639a_1440w.webp"></p><p>比如我们想知道整体市场的规模，增长率和CAGR（年复合增长率），根据我们上方对DATE字段的分析，可以做如下操作：</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">def</span> <span class="title function_">get_kpi</span>(<span class="params">df</span>):</span><br><span class="line"></span><br><span class="line">    <span class="comment"># 按列求和为市场总值的Series</span></span><br><span class="line">    market_total = df.<span class="built_in">sum</span>(axis=<span class="number">1</span>)</span><br><span class="line">    <span class="comment"># 最后一行（最后一个DATE）就是最新的市场规模</span></span><br><span class="line">    market_size = market_total.iloc[-<span class="number">1</span>]</span><br><span class="line">    <span class="comment"># 市场按列求和，倒数第5行（倒数第5个DATE）就是同比的市场规模，可以用来求同比增长率</span></span><br><span class="line">    market_gr = market_total.iloc[-<span class="number">1</span>] / market_total.iloc[-<span class="number">5</span>] - <span class="number">1</span></span><br><span class="line">    <span class="comment"># 因为数据第一年是四年前的同期季度，时间序列收尾相除后开四次方根可得到年复合增长率</span></span><br><span class="line">    market_cagr = (market_total.iloc[-<span class="number">1</span>] / market_total.iloc[<span class="number">0</span>]) ** (<span class="number">0.25</span>) - <span class="number">1</span></span><br><span class="line">    <span class="keyword">if</span> market_size == np.inf <span class="keyword">or</span> market_size == -np.inf:</span><br><span class="line">        market_size = <span class="string">&quot;N/A&quot;</span></span><br><span class="line">    <span class="keyword">if</span> market_gr == np.inf <span class="keyword">or</span> market_gr == -np.inf:</span><br><span class="line">        market_gr = <span class="string">&quot;N/A&quot;</span></span><br><span class="line">    <span class="keyword">if</span> market_cagr == np.inf <span class="keyword">or</span> market_cagr == -np.inf:</span><br><span class="line">        market_cagr = <span class="string">&quot;N/A&quot;</span></span><br><span class="line">        </span><br><span class="line">    <span class="keyword">return</span> &#123;</span><br><span class="line">        <span class="string">&quot;market_size&quot;</span>: market_size,</span><br><span class="line">        <span class="string">&quot;market_gr&quot;</span>: market_gr,</span><br><span class="line">        <span class="string">&quot;market_cagr&quot;</span>: market_cagr,</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line"><span class="built_in">print</span>(get_kpi(pivoted))</span><br></pre></td></tr></table></figure><p>结果为：<br><img src="/images/python-djangosqlpand/v2-ab876ca2b814b7b4c5c2db483846ebe6_1440w.webp"></p><p><img src="/images/python-djangosqlpand/v2-ab876ca2b814b7b4c5c2db483846ebe6_1440w.webp"></p><p>ARB市场的年销售额为60亿，年增长率+3.1%，年复合增长率+5.2%</p><p>我们可以继续以这个思路生成更复杂的结果：</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">def</span> <span class="title function_">ptable</span>(<span class="params">df</span>):</span><br><span class="line">    <span class="comment"># 份额</span></span><br><span class="line">    df_share = df.transform(<span class="keyword">lambda</span> x: x/x.<span class="built_in">sum</span>(), axis=<span class="number">1</span>)</span><br><span class="line">    </span><br><span class="line">    <span class="comment"># 同比增长率，要考虑分子为0的问题</span></span><br><span class="line">    df_gr = df.pct_change(periods=<span class="number">4</span>)</span><br><span class="line">    df_gr.dropna(how=<span class="string">&#x27;all&#x27;</span>,inplace=<span class="literal">True</span>)</span><br><span class="line">    df_gr.replace([np.inf, -np.inf], np.nan, inplace=<span class="literal">True</span>)</span><br><span class="line">    </span><br><span class="line">    <span class="comment"># 最新滚动年绝对值表现及同比净增长</span></span><br><span class="line">    df_latest = df.iloc[-<span class="number">1</span>,:]</span><br><span class="line">    df_latest_diff = df.iloc[-<span class="number">1</span>,:] - df.iloc[-<span class="number">5</span>,:]</span><br><span class="line"></span><br><span class="line">    <span class="comment"># 最新滚动年份额表现及同比份额净增长</span></span><br><span class="line">    df_share_latest = df_share.iloc[-<span class="number">1</span>, :]</span><br><span class="line">    df_share_latest_diff = df_share.iloc[-<span class="number">1</span>, :] - df_share.iloc[-<span class="number">5</span>, :]</span><br><span class="line">    </span><br><span class="line">    <span class="comment"># 进阶指标EI，衡量与市场增速的对比，高于100则为跑赢大盘</span></span><br><span class="line">    df_gr_latest = df_gr.iloc[-<span class="number">1</span>,:]</span><br><span class="line">    df_total_gr_latest = df.<span class="built_in">sum</span>(axis=<span class="number">1</span>).iloc[-<span class="number">1</span>]/df.<span class="built_in">sum</span>(axis=<span class="number">1</span>).iloc[-<span class="number">5</span>] -<span class="number">1</span></span><br><span class="line">    df_ei_latest = (df_gr_latest+<span class="number">1</span>)/(df_total_gr_latest+<span class="number">1</span>)*<span class="number">100</span></span><br><span class="line"></span><br><span class="line">    df_combined = pd.concat([df_latest, df_latest_diff, df_share_latest, df_share_latest_diff, df_gr_latest, df_ei_latest], axis=<span class="number">1</span>)</span><br><span class="line">    df_combined.columns = [<span class="string">&#x27;最新滚动年销售额&#x27;</span>,</span><br><span class="line">                           <span class="string">&#x27;净增长&#x27;</span>,</span><br><span class="line">                           <span class="string">&#x27;份额&#x27;</span>,</span><br><span class="line">                           <span class="string">&#x27;份额同比变化&#x27;</span>,</span><br><span class="line">                           <span class="string">&#x27;同比增长率&#x27;</span>,</span><br><span class="line">                           <span class="string">&#x27;EI&#x27;</span>]</span><br><span class="line"></span><br><span class="line">    <span class="keyword">return</span> df_combined</span><br><span class="line"><span class="built_in">print</span>(ptable(pivoted))</span><br></pre></td></tr></table></figure><p>市场内各通用名分子的表现一览</p><p>把以上数据处理过程整合入views.py里的index方法，此时我们可以有更丰富的context传入display.html模板：</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">def</span> <span class="title function_">index</span>(<span class="params">request</span>):</span><br><span class="line">    sql = sqlparse(<span class="string">&#x27;MAT&#x27;</span>, <span class="string">&#x27;Value&#x27;</span>, <span class="string">&quot; [TC III] = &#x27;C09C ANGIOTENS-II ANTAG, PLAIN|血管紧张素II拮抗剂，单一用药&#x27;&quot;</span>) <span class="comment">#读取ARB市场的滚动年销售额数据</span></span><br><span class="line">    df = pd.read_sql_query(sql, ENGINE)  <span class="comment"># 将sql语句结果读取至Pandas Dataframe</span></span><br><span class="line"></span><br><span class="line">    pivoted = pd.pivot_table(df,</span><br><span class="line">                           values=<span class="string">&#x27;AMOUNT&#x27;</span>,  <span class="comment"># 数据透视汇总值为AMOUNT字段，一般保持不变</span></span><br><span class="line">                           index=<span class="string">&#x27;DATE&#x27;</span>,  <span class="comment"># 数据透视行为DATE字段，一般保持不变</span></span><br><span class="line">                           columns=<span class="string">&#x27;MOLECULE&#x27;</span>,  <span class="comment"># 数据透视列为MOLECULE字段，该字段以后应跟随分析需要动态传参</span></span><br><span class="line">                           aggfunc=np.<span class="built_in">sum</span>) <span class="comment"># 数据透视汇总方式为求和，一般保持不变</span></span><br><span class="line">    <span class="keyword">if</span> pivoted.empty <span class="keyword">is</span> <span class="literal">False</span>:</span><br><span class="line">        pivoted.sort_values(by=pivoted.index[-<span class="number">1</span>], axis=<span class="number">1</span>, ascending=<span class="literal">False</span>, inplace=<span class="literal">True</span>) <span class="comment">#结果按照最后一个DATE表现排序</span></span><br><span class="line"></span><br><span class="line">    <span class="comment"># KPI</span></span><br><span class="line">    kpi = get_kpi(pivoted)</span><br><span class="line"></span><br><span class="line">    context = &#123;</span><br><span class="line">        <span class="string">&quot;market_size&quot;</span>: kpi[<span class="string">&quot;market_size&quot;</span>],</span><br><span class="line">        <span class="string">&quot;market_gr&quot;</span>: kpi[<span class="string">&quot;market_gr&quot;</span>],</span><br><span class="line">        <span class="string">&quot;market_cagr&quot;</span>: kpi[<span class="string">&quot;market_cagr&quot;</span>],</span><br><span class="line">        <span class="string">&#x27;ptable&#x27;</span>: ptable(pivoted).to_html()</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="keyword">return</span> render(request, <span class="string">&#x27;chpa_data/display.html&#x27;</span>, context)</span><br></pre></td></tr></table></figure><p>在前端display.html模板里写入对应context的django tag，并用Semantic UI的tab语法稍作布局。</p><figure class="highlight html"><table><tr><td class="code"><pre><span class="line">&#123;<span class="symbol">&amp;#37;</span> extends &quot;chpa_data/analysis.html&quot; <span class="symbol">&amp;#37;</span>&#125;</span><br><span class="line"></span><br><span class="line">&#123;<span class="symbol">&amp;#37;</span> block display <span class="symbol">&amp;#37;</span>&#125;</span><br><span class="line"><span class="comment">&lt;!-- 创建2个Semantic UI tab，根据鼠标点击切换，以保证页面干净清爽 --&gt;</span></span><br><span class="line"><span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;ui pointing secondary menu&quot;</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">a</span> <span class="attr">class</span>=<span class="string">&quot;item active&quot;</span> <span class="attr">data-tab</span>=<span class="string">&quot;total&quot;</span>&gt;</span><span class="tag">&lt;<span class="name">i</span> <span class="attr">class</span>=<span class="string">&quot;circle icon&quot;</span>&gt;</span><span class="tag">&lt;/<span class="name">i</span>&gt;</span>总体表现<span class="tag">&lt;/<span class="name">a</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">a</span> <span class="attr">class</span>=<span class="string">&quot;item&quot;</span> <span class="attr">data-tab</span>=<span class="string">&quot;competition&quot;</span>&gt;</span><span class="tag">&lt;<span class="name">i</span> <span class="attr">class</span>=<span class="string">&quot;trophy icon&quot;</span>&gt;</span><span class="tag">&lt;/<span class="name">i</span>&gt;</span>竞争现状<span class="tag">&lt;/<span class="name">a</span>&gt;</span></span><br><span class="line"><span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line"><span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;ui tab segment active&quot;</span> <span class="attr">data-tab</span>=<span class="string">&quot;total&quot;</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">h3</span> <span class="attr">class</span>=<span class="string">&quot;ui header&quot;</span>&gt;</span></span><br><span class="line">        <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;content&quot;</span>&gt;</span></span><br><span class="line">            定义市场当前表现</span><br><span class="line">            <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;sub header&quot;</span>&gt;</span>KPIs<span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">        <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;/<span class="name">h3</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;ui divider&quot;</span>&gt;</span><span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">    <span class="comment">&lt;!-- Semantic UI的statistic类能呈现“醒目大数字”的效果 --&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;ui small three statistics&quot;</span>&gt;</span></span><br><span class="line">        <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;statistic&quot;</span>&gt;</span></span><br><span class="line">            <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;value&quot;</span> <span class="attr">id</span>=<span class="string">&quot;value_size&quot;</span>&gt;</span></span><br><span class="line">                &#123;<span class="symbol">&amp;#123;</span> market_size <span class="symbol">&amp;#125;</span>&#125;</span><br><span class="line">            <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">            <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;label&quot;</span> <span class="attr">id</span>=<span class="string">&quot;label_size_unit&quot;</span>&gt;</span></span><br><span class="line">                滚动年金额</span><br><span class="line">            <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">        <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">        <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;statistic&quot;</span> <span class="attr">id</span>=<span class="string">&quot;div_gr&quot;</span>&gt;</span></span><br><span class="line">            <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;value&quot;</span> <span class="attr">id</span>=<span class="string">&quot;value_gr&quot;</span>&gt;</span></span><br><span class="line">                &#123;<span class="symbol">&amp;#123;</span> market_gr <span class="symbol">&amp;#125;</span>&#125;</span><br><span class="line">            <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">            <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;label&quot;</span>&gt;</span></span><br><span class="line">                同比增长</span><br><span class="line">            <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">        <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">        <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;statistic&quot;</span> <span class="attr">id</span>=<span class="string">&quot;div_cagr&quot;</span>&gt;</span></span><br><span class="line">            <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;value&quot;</span> <span class="attr">id</span>=<span class="string">&quot;value_cagr&quot;</span>&gt;</span></span><br><span class="line">                &#123;<span class="symbol">&amp;#123;</span> market_cagr <span class="symbol">&amp;#125;</span>&#125;</span><br><span class="line">            <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">            <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;label&quot;</span>&gt;</span></span><br><span class="line">                4年CAGR</span><br><span class="line">            <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">        <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line"><span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line"><span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;ui tab segment&quot;</span> <span class="attr">data-tab</span>=<span class="string">&quot;competition&quot;</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">h3</span> <span class="attr">class</span>=<span class="string">&quot;ui header&quot;</span>&gt;</span></span><br><span class="line">        <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;content&quot;</span>&gt;</span></span><br><span class="line">            最新横断面KPI一览</span><br><span class="line">            <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;sub header&quot;</span>&gt;</span>数据表格<span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">        <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;/<span class="name">h3</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;ui divider&quot;</span>&gt;</span><span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line"></span><br><span class="line">    <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;ui container&quot;</span> <span class="attr">id</span>=<span class="string">&#x27;result_table&#x27;</span> <span class="attr">style</span>=<span class="string">&quot;width: 100%; overflow-x: scroll; overflow-y: hidden;&quot;</span>&gt;</span></span><br><span class="line">        <span class="comment">&lt;!-- Django渲染html代码时需要加入|safe，保证html不会被自动转义 --&gt;</span></span><br><span class="line">        &#123;<span class="symbol">&amp;#123;</span> ptable|safe <span class="symbol">&amp;#125;</span>&#125;</span><br><span class="line">    <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line"></span><br><span class="line"><span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line"></span><br><span class="line"><span class="comment">&lt;!-- 下方js为保证Semantic UI tab类工作 --&gt;</span></span><br><span class="line"><span class="tag">&lt;<span class="name">script</span>&gt;</span><span class="language-javascript"></span></span><br><span class="line"><span class="language-javascript">    $(<span class="string">&#x27;.pointing.secondary.menu .item&#x27;</span>).<span class="title function_">tab</span>();</span></span><br><span class="line"><span class="language-javascript"></span><span class="tag">&lt;/<span class="name">script</span>&gt;</span></span><br><span class="line">&#123;<span class="symbol">&amp;#37;</span> endblock <span class="symbol">&amp;#37;</span>&#125;</span><br></pre></td></tr></table></figure><p>因为css框架的语法都是应用类的比较简单，在本系列文章我较少讲解Semantic UI，后续有不少朋友反应这里有个坑需要注意：</p><p><strong>要保证Semantic UI的tab功能正常，需要保证ui pointing secondary menu这个Div下的item和后续对应的ui tab segment，两者的data-tab属性一一对应，并且两者都有且只有一个active。</strong></p><p>现在再在浏览器里访问127.0.0.1:8088&#x2F;chpa&#x2F;index，界面会变为：<br><img src="/images/python-djangosqlpand/v2-d992ed4977bbbcae9b4bf1943249d60e_1440w.webp"></p><p>如果您使用了我提供的随机dummy数据测试，这里结果不一样是正常的，因为我写此文时使用了原始真实数据<br><img src="/images/python-djangosqlpand/v2-3a1f24e1a7551129051288e8db1894e4_1440w.webp"></p><p>点击“竞争现状”那个tab，会出现我们渲染过来的ptable：</p><p>如果您使用了我提供的随机dummy数据测试，这里结果不一样是正常的，因为我写此文时使用了原始真实数据</p><p>至此我们已经初步取得了我们想要的数据分析结果，除了更丰富的分析维度，我们还缺少交互、格式化和可视化，这些留待以后讨论。</p><p>下一篇请移步：</p>]]>
    </content>
    <id>https://blog.malu.tech/python-django-sql-pandas-pyecharts-data-analysis-platform-4/</id>
    <link href="https://blog.malu.tech/python-django-sql-pandas-pyecharts-data-analysis-platform-4/"/>
    <published>2024-01-21T16:00:00.000Z</published>
    <summary>Python Django+SQL+Pandas+Pyecharts自建在线数据分析平台系列文章第四篇：SQL+Pandas初步处理数据</summary>
    <title>Python Django+SQL+Pandas+Pyecharts自建在线数据分析平台（四）</title>
    <updated>2024-01-21T16:00:00.000Z</updated>
  </entry>
  <entry>
    <author>
      <name>Road</name>
    </author>
    <category term="Python" scheme="https://blog.malu.tech/categories/Python/"/>
    <category term="Django" scheme="https://blog.malu.tech/categories/Python/Django/"/>
    <category term="数据分析" scheme="https://blog.malu.tech/categories/Python/Django/%E6%95%B0%E6%8D%AE%E5%88%86%E6%9E%90/"/>
    <category term="Python" scheme="https://blog.malu.tech/tags/Python/"/>
    <category term="Django" scheme="https://blog.malu.tech/tags/Django/"/>
    <category term="SQL" scheme="https://blog.malu.tech/tags/SQL/"/>
    <category term="Pandas" scheme="https://blog.malu.tech/tags/Pandas/"/>
    <category term="Pyecharts" scheme="https://blog.malu.tech/tags/Pyecharts/"/>
    <category term="数据分析平台" scheme="https://blog.malu.tech/tags/%E6%95%B0%E6%8D%AE%E5%88%86%E6%9E%90%E5%B9%B3%E5%8F%B0/"/>
    <content>
      <![CDATA[<blockquote><p><strong>文章来源</strong>：<a href="https://zhuanlan.zhihu.com/p/142490087">Python Django+SQL+Pandas+Pyecharts自建在线数据分析平台（一）</a><br><strong>作者</strong>：<a href="https://www.zhihu.com/people/chen-cheng-76-40">ccpic</a><br><strong>感谢</strong>：感谢作者 ccpic 分享的优质内容，本网页主要用于学习知识的存档备份，欢迎点击原网页支持作者。</p></blockquote><p>本篇是系列文章的第三篇：</p><p>（一）需求分析&amp;技术实现</p><p>（二）初步搭建Django环境</p><p>（四）SQL+Pandas初步处理数据</p><p>（五）前端表单交互</p><p>（六）Ajax异步传参与加载</p><p>（七）前端数据格式的处理</p><p>（八）DataTables接管前端表格</p><p>（九）Pyecharts实现交互图表</p><p>（十）静态图表的展示</p><p>（十一）“导出数据至Excel”功能</p><p>（十二）添加和配置缓存</p><p>（十三）用户登录系统</p><p>（十四）部署Django至生产环境</p><p>在上一章初步打通前后端通信后，接下来的方向无疑是两端的分别扩展和完善。本章先来看一下前端页面布局和Django模板的问题。</p><p>本例的页面布局没有任何标新立异的地方，主页面采用header-body-footer的上中下布局，header和footer是几乎静态的，body则是动态的。body在部分功能下是一块整体，在核心的分析功能下分成filter-display的左右布局。filter承担向后端提交input的角色，display则展示后端根据input返回的response。</p><p>（在更进阶的情况下，负责分析的body页面还可以进一步分为filter-display-setting的左中右布局）。</p><p>我们构思的整个布局如下图所示：<br><img src="/images/python-djangosqlpand/v2-5898937c24db49e7d2ad5ab41ee3c9c2_1440w.webp"></p><p>根据此设计，我们需要搭建相应的Django模板架构，并充分利用其模板继承特性，其中主要需要理解base-extend机制以及block和include两个主要的模板语句。</p><p>在这里我省略了讲解Django模板加载器的部分，其实这一部分也是有讲究的，尤其是对于多个app的更大型的工程。</p><p><strong>按照官网推荐的结构，先在本例的chpa_data新建文件夹templates，再在templates内新建和app同名的文件夹chpa_data，再在其内新建Django的html模板文件。</strong><br><img src="/images/python-djangosqlpand/v2-3cdb4f5900228672ab1fec6e36c2c631_1440w.webp"></p><p>首先，创建一个base.html代表这个模板的基础结构（可以称之为基础模板或父模板）：</p><figure class="highlight html"><table><tr><td class="code"><pre><span class="line"><span class="comment">&lt;!-- 载入静态文件 更早版本可能会用load staticfiles --&gt;</span></span><br><span class="line">&#123;<span class="symbol">&amp;#37;</span> load static <span class="symbol">&amp;#37;</span>&#125;</span><br><span class="line"></span><br><span class="line">![](images/python-djangosqlpand/v2-44934e23627fc7a9c1828acfbc8376d5_1440w.webp)</span><br><span class="line"><span class="meta">&lt;!DOCTYPE <span class="keyword">html</span>&gt;</span></span><br><span class="line"><span class="comment">&lt;!-- 网站主语言 --&gt;</span></span><br><span class="line"><span class="tag">&lt;<span class="name">html</span> <span class="attr">lang</span>=<span class="string">&quot;zh-cn&quot;</span>&gt;</span></span><br><span class="line"></span><br><span class="line"><span class="tag">&lt;<span class="name">head</span>&gt;</span></span><br><span class="line">    <span class="comment">&lt;!-- 网站采用的字符编码 --&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">meta</span> <span class="attr">charset</span>=<span class="string">&quot;utf-8&quot;</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">title</span>&gt;</span>数据分析<span class="tag">&lt;/<span class="name">title</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">link</span> <span class="attr">rel</span>=<span class="string">&quot;stylesheet&quot;</span> <span class="attr">href</span>=<span class="string">&quot;&#123;<span class="symbol">&amp;#37;</span> static &#x27;datatables/dataTables.semanticui.min.css&#x27; <span class="symbol">&amp;#37;</span>&#125;&quot;</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">link</span> <span class="attr">rel</span>=<span class="string">&quot;stylesheet&quot;</span> <span class="attr">href</span>=<span class="string">&quot;&#123;<span class="symbol">&amp;#37;</span> static &#x27;Semantic-UI-CSS-master/semantic.min.css&#x27; <span class="symbol">&amp;#37;</span>&#125;&quot;</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">script</span> <span class="attr">src</span>=<span class="string">&quot;&#123;<span class="symbol">&amp;#37;</span> static &#x27;jquery/jquery-3.4.1.js&#x27; <span class="symbol">&amp;#37;</span>&#125;&quot;</span>&gt;</span><span class="tag">&lt;/<span class="name">script</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">script</span> <span class="attr">src</span>=<span class="string">&quot;&#123;<span class="symbol">&amp;#37;</span> static &#x27;datatables/jquery.dataTables.min.js&#x27; <span class="symbol">&amp;#37;</span>&#125;&quot;</span>&gt;</span><span class="tag">&lt;/<span class="name">script</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">script</span> <span class="attr">src</span>=<span class="string">&quot;&#123;<span class="symbol">&amp;#37;</span> static &#x27;datatables/dataTables.semanticui.min.js&#x27; <span class="symbol">&amp;#37;</span>&#125;&quot;</span>&gt;</span><span class="tag">&lt;/<span class="name">script</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">script</span> <span class="attr">src</span>=<span class="string">&quot;&#123;<span class="symbol">&amp;#37;</span> static &#x27;datatables/percentageBars.js&#x27; <span class="symbol">&amp;#37;</span>&#125;&quot;</span>&gt;</span><span class="tag">&lt;/<span class="name">script</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">script</span> <span class="attr">type</span>=<span class="string">&quot;text/javascript&quot;</span> <span class="attr">src</span>=<span class="string">&quot;&#123;<span class="symbol">&amp;#37;</span> static &#x27;Semantic-UI-CSS-master/semantic.js&#x27; <span class="symbol">&amp;#37;</span>&#125;&quot;</span>&gt;</span><span class="tag">&lt;/<span class="name">script</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">script</span> <span class="attr">src</span>=<span class="string">&quot;&#123;<span class="symbol">&amp;#37;</span> static &#x27;echarts/echarts.min.js&#x27; <span class="symbol">&amp;#37;</span>&#125;&quot;</span>&gt;</span><span class="tag">&lt;/<span class="name">script</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">script</span> <span class="attr">src</span>=<span class="string">&quot;&#123;<span class="symbol">&amp;#37;</span> static &#x27;echarts/china.js&#x27; <span class="symbol">&amp;#37;</span>&#125;&quot;</span>&gt;</span><span class="tag">&lt;/<span class="name">script</span>&gt;</span></span><br><span class="line"><span class="tag">&lt;/<span class="name">head</span>&gt;</span></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="tag">&lt;<span class="name">body</span>&gt;</span></span><br><span class="line"></span><br><span class="line"><span class="comment">&lt;!-- 引入导航栏 --&gt;</span></span><br><span class="line">&#123;<span class="symbol">&amp;#37;</span> include &#x27;chpa_data/header.html&#x27; <span class="symbol">&amp;#37;</span>&#125;</span><br><span class="line"><span class="comment">&lt;!-- 预留具体页面的位置 --&gt;</span></span><br><span class="line">&#123;<span class="symbol">&amp;#37;</span> block body <span class="symbol">&amp;#37;</span>&#125;&#123;<span class="symbol">&amp;#37;</span> endblock body <span class="symbol">&amp;#37;</span>&#125;</span><br><span class="line"><span class="comment">&lt;!-- 引入注脚 --&gt;</span></span><br><span class="line">&#123;<span class="symbol">&amp;#37;</span> include &#x27;chpa_data/footer.html&#x27; <span class="symbol">&amp;#37;</span>&#125;</span><br><span class="line"></span><br><span class="line"><span class="tag">&lt;/<span class="name">body</span>&gt;</span></span><br><span class="line"></span><br><span class="line"><span class="tag">&lt;/<span class="name">html</span>&gt;</span></span><br></pre></td></tr></table></figure><p>在header部分我们引用了这个工程需要的静态文件，所有继承base的页面就可以不再引用了。这里需要在datasite下的settings.py加一条设置以识别静态文件夹的位置：</p><figure class="highlight django"><table><tr><td class="code"><pre><span class="line"><span class="language-xml">STATIC_URL = &#x27;/static/&#x27; # 静态文件夹相对于工程根目录的相对位置</span></span><br><span class="line"><span class="language-xml">STATICFILES_DIRS = (</span></span><br><span class="line"><span class="language-xml">    os.path.join(BASE_DIR, &quot;static&quot;), </span></span><br><span class="line"><span class="language-xml">)  # 该语句建议保留，对低版本的Django也是指明静态文件的位置，后续版本有功能改变</span></span><br></pre></td></tr></table></figure><p>而base.html代码中最重要的是需要理解 {&#37; include &#37;}和{&#37; block &#37;}{&#37; endblock &#37;}的区别。{&#37; include &#37;}是直接引用一个相对独立的代码块，{&#37; block &#37;}{&#37; endblock &#37;}则是给所有extend base.html的子模板预留可以复写的位置，在子模板要有对应的block。从另一个角度说，子模版block之间的内容可以覆盖父模板中的相同block。</p><p><strong>这里的概念有些容易混淆，其实这么理解就可以了，当代码块是通用的相对静态的时候，使用include语句直接引用；而当可能有多个子模板多次继承重写同一部分时，预留block。或者简单说，include是多对一的关系（但因为继承，大部分实际情况下是一对一），而block是一对多的关系。</strong></p><p>所以对于我们的app，header和footer是相对静态的，不会出现多个不同的header和footer。我们在templates文件夹下创建header.html和footer.html，只是为了方便单独管理。使用semantic ui的语句简单写一点header和footer。</p><p>header.html:</p><figure class="highlight html"><table><tr><td class="code"><pre><span class="line"><span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;ui large top fixed inverted menu&quot;</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;ui container&quot;</span>&gt;</span></span><br><span class="line">        <span class="tag">&lt;<span class="name">a</span> <span class="attr">href</span>=<span class="string">&#123;&amp;#37;</span> <span class="attr">url</span> &quot;<span class="attr">chpa:index</span>&quot; &amp;#<span class="attr">37</span>;&#125; <span class="attr">class</span>=<span class="string">&quot;item&quot;</span>&gt;</span>首页<span class="tag">&lt;/<span class="name">a</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line"><span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br></pre></td></tr></table></figure><p>这里第一次出现Django特色的url tag，{&#37;url”chpa:index”&#37;}的结构实际是对应chpa_data的urls.py文件里的{&#37; url “app_name:view_name” &#37;}。一定注意这里是urls.py里为urls们命名的app_name参数，而不是实际上这个工程里app的name。所以这里是chpa而不是chpa_data。<br><img src="/images/python-djangosqlpand/v2-6ce74aac762ccd83497e3674e609daba_1440w.webp"></p><p>footer.html:</p><figure class="highlight html"><table><tr><td class="code"><pre><span class="line"><span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;ui inverted vertical footer segment&quot;</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;ui center aligned container&quot;</span>&gt;</span></span><br><span class="line">        占位</span><br><span class="line">    <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line"><span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br></pre></td></tr></table></figure><p>注意header和footer的代码就是这么短，是不需要extends语句的。</p><p>而base里预留的{&#37; block body &#37;}{&#37; endblock body &#37;}部分是可以复用的，我们可以测试两个页面分别继承的情况，这两个页面的开头语句都是：</p><figure class="highlight django"><table><tr><td class="code"><pre><span class="line"><span class="language-xml">&#123;<span class="symbol">&amp;#37;</span> extends &quot;chpa_data/base.html&quot; <span class="symbol">&amp;#37;</span>&#125;</span></span><br></pre></td></tr></table></figure><p>我们可以有一个静态的项目首页index.html，他的内容是极其简单的，以后可以是个含有导航信息或说明文字的静态首页，这次我们先预留一个{&#123; data &#125;} tag准备接受测试时后台传来的数据：</p><figure class="highlight django"><table><tr><td class="code"><pre><span class="line"><span class="language-xml"><span class="comment">&lt;!-- extends表明此页面继承自 base.html 文件 --&gt;</span></span></span><br><span class="line"><span class="language-xml">&#123;<span class="symbol">&amp;#37;</span> extends &quot;chpa_data/base.html&quot; <span class="symbol">&amp;#37;</span>&#125;</span></span><br><span class="line"><span class="language-xml"></span></span><br><span class="line"><span class="language-xml"><span class="comment">&lt;!-- 隐藏分隔条 --&gt;</span></span></span><br><span class="line"><span class="language-xml"><span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;ui hidden divider&quot;</span>&gt;</span><span class="tag">&lt;/<span class="name">div</span>&gt;</span></span></span><br><span class="line"><span class="language-xml"></span></span><br><span class="line"><span class="language-xml"><span class="comment">&lt;!-- 写入 base.html 中定义的 body content --&gt;</span></span></span><br><span class="line"><span class="language-xml">&#123;<span class="symbol">&amp;#37;</span> block body <span class="symbol">&amp;#37;</span>&#125;</span></span><br><span class="line"><span class="language-xml"><span class="comment">&lt;!-- 因为semantic ui的特性需要在body部分写个pusher避免内容被顶部导航栏遮挡 --&gt;</span></span></span><br><span class="line"><span class="language-xml"><span class="tag">&lt;<span class="name">div</span> <span class="attr">id</span>=<span class="string">&quot;pusher&quot;</span> <span class="attr">class</span>=<span class="string">&quot;pusher&quot;</span> <span class="attr">style</span>=<span class="string">&quot;padding-top:50px&quot;</span>&gt;</span></span></span><br><span class="line"><span class="language-xml">    &#123;<span class="symbol">&amp;#123;</span> data <span class="symbol">&amp;#125;</span>&#125;</span></span><br><span class="line"><span class="language-xml"><span class="tag">&lt;/<span class="name">div</span>&gt;</span></span></span><br><span class="line"><span class="language-xml">&#123;<span class="symbol">&amp;#37;</span> endblock body <span class="symbol">&amp;#37;</span>&#125;</span></span><br></pre></td></tr></table></figure><p>再创建一个相对复杂的analysis.html，根据本章开头的设计，这个页面应该是左边是filter，而右边是display。这里涉及到进一步嵌套的模板继承，filter一般不会变，所以直接include，而display有可能有多个子模板再复用，继续预留block位置：</p><figure class="highlight django"><table><tr><td class="code"><pre><span class="line"><span class="language-xml"><span class="comment">&lt;!-- extends表明此页面继承自 base.html 文件 --&gt;</span></span></span><br><span class="line"><span class="language-xml">&#123;<span class="symbol">&amp;#37;</span> extends &quot;chpa_data/base.html&quot; <span class="symbol">&amp;#37;</span>&#125;</span></span><br><span class="line"><span class="language-xml"></span></span><br><span class="line"><span class="language-xml"><span class="comment">&lt;!-- 隐藏分隔条 --&gt;</span></span></span><br><span class="line"><span class="language-xml"><span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;ui hidden divider&quot;</span>&gt;</span><span class="tag">&lt;/<span class="name">div</span>&gt;</span></span></span><br><span class="line"><span class="language-xml"></span></span><br><span class="line"><span class="language-xml"><span class="comment">&lt;!-- 写入 base.html 中定义的 body content --&gt;</span></span></span><br><span class="line"><span class="language-xml">&#123;<span class="symbol">&amp;#37;</span> block body <span class="symbol">&amp;#37;</span>&#125;</span></span><br><span class="line"><span class="language-xml"><span class="comment">&lt;!-- 因为semantic ui的特性需要在body部分写个pusher避免内容被顶部导航栏遮挡 --&gt;</span></span></span><br><span class="line"><span class="language-xml"><span class="tag">&lt;<span class="name">div</span> <span class="attr">id</span>=<span class="string">&quot;pusher&quot;</span> <span class="attr">class</span>=<span class="string">&quot;pusher&quot;</span> <span class="attr">style</span>=<span class="string">&quot;padding-top:50px&quot;</span>&gt;</span></span></span><br><span class="line"><span class="language-xml">    <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;ui celled grid&quot;</span>&gt;</span></span></span><br><span class="line"><span class="language-xml">        <span class="comment">&lt;!-- 左边3/16为filter页面, include --&gt;</span></span></span><br><span class="line"><span class="language-xml">        <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;three wide column&quot;</span>&gt;</span></span></span><br><span class="line"><span class="language-xml">            &#123;<span class="symbol">&amp;#37;</span> include &#x27;chpa_data/filter.html&#x27; <span class="symbol">&amp;#37;</span>&#125;</span></span><br><span class="line"><span class="language-xml">        <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span></span><br><span class="line"><span class="language-xml"></span></span><br><span class="line"><span class="language-xml">        <span class="comment">&lt;!-- 右边13/16为display页面, 预留block --&gt;</span></span></span><br><span class="line"><span class="language-xml">        <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">&quot;thirteen wide column&quot;</span>&gt;</span></span></span><br><span class="line"><span class="language-xml">            &#123;<span class="symbol">&amp;#37;</span> block display <span class="symbol">&amp;#37;</span>&#125;&#123;<span class="symbol">&amp;#37;</span> endblock display <span class="symbol">&amp;#37;</span>&#125;</span></span><br><span class="line"><span class="language-xml">        <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span></span><br><span class="line"><span class="language-xml">    <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span></span><br><span class="line"><span class="language-xml"><span class="tag">&lt;/<span class="name">div</span>&gt;</span></span></span><br><span class="line"><span class="language-xml">&#123;<span class="symbol">&amp;#37;</span> endblock body <span class="symbol">&amp;#37;</span>&#125;</span></span><br></pre></td></tr></table></figure><p>最后再创建filter.html文件，可以暂时为空。</p><p>**display.html文件，注意这里的第一行不是extends base.html而是extends analysis.html。**我们再和index.html一样预留 {&#123; data &#125;} tag准备测试：</p><figure class="highlight django"><table><tr><td class="code"><pre><span class="line"><span class="language-xml">&#123;<span class="symbol">&amp;#37;</span> extends &quot;chpa_data/analysis.html&quot; <span class="symbol">&amp;#37;</span>&#125;</span></span><br><span class="line"><span class="language-xml">&#123;<span class="symbol">&amp;#37;</span> block display <span class="symbol">&amp;#37;</span>&#125;</span></span><br><span class="line"><span class="language-xml">    &#123;<span class="symbol">&amp;#123;</span> data <span class="symbol">&amp;#125;</span>&#125;</span></span><br><span class="line"><span class="language-xml">&#123;<span class="symbol">&amp;#37;</span> endblock display <span class="symbol">&amp;#37;</span>&#125;</span></span><br></pre></td></tr></table></figure><p>此时Django模板架构完毕，再确认一次项目结构：<br><img src="/images/python-djangosqlpand/v2-3cdb4f5900228672ab1fec6e36c2c631_1440w.webp"></p><p>我们可以修改上一章中views.py的index方法做个测试，因为在模板里预留了{&#123; data &#125;}tag，我们不希望再传一个整体的HttpResponse了，而是希望传一个包含data键值的context。修改代码如下：</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">from</span> django.shortcuts <span class="keyword">import</span> render</span><br><span class="line"></span><br><span class="line">...</span><br><span class="line"></span><br><span class="line"><span class="keyword">def</span> <span class="title function_">index</span>(<span class="params">request</span>):</span><br><span class="line">    sql = <span class="string">&quot;Select count(*) from data&quot;</span> <span class="comment">#标准sql语句，此处为测试返回数据库data表的数据条目n，之后可以用python处理字符串的方式动态扩展</span></span><br><span class="line">    df = pd.read_sql_query(sql, ENGINE) <span class="comment">#将sql语句结果读取至Pandas Dataframe</span></span><br><span class="line">    context = &#123;<span class="string">&#x27;data&#x27;</span>: df &#125;</span><br><span class="line">    <span class="keyword">return</span> render(request, <span class="string">&#x27;chpa_data/index.html&#x27;</span>, context)</span><br></pre></td></tr></table></figure><p>可以看到首页已经发生了改变。<br><img src="/images/python-djangosqlpand/v2-719011a88212df3a4961db90b3320de8_1440w.webp"></p><p><img src="/images/python-djangosqlpand/v2-719011a88212df3a4961db90b3320de8_1440w.webp"></p><p>如果render到display模板会怎样呢？</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">return</span> render(request, <span class="string">&#x27;chpa_data/display.html&#x27;</span>, context)</span><br></pre></td></tr></table></figure><p><img src="/images/python-djangosqlpand/v2-44934e23627fc7a9c1828acfbc8376d5_1440w.webp"></p><p>很明显我们为分析功能设计的filter-display左右布局也生效了。</p><p>本章是反馈出问题最多的地方，特再次总结一遍容易踩坑的要点：</p><ol><li><strong>项目结构建议完全按照截图。在教程的早期版本我曾经把模板html文件直接放在templates文件夹下，后来一些朋友反馈Django高级版本会报错，所以请不要这么做。</strong></li><li><strong>按照截图推荐的结构的情况下，所有要引用模板path的extends, include和render等语句，路径一定都是’chpa_data&#x2F;xxx.html’而不是’xxx.html’。这是因为加载模板是根据绝对位置（templates文件夹）而不是相对位置。</strong></li><li><strong>只有预留block的子模板要在头部加入对应的extends语句，include的子模版不需要加。</strong></li></ol><p>下一篇SQL+Pandas初步处理数据，见：</p>]]>
    </content>
    <id>https://blog.malu.tech/python-django-sql-pandas-pyecharts-data-analysis-platform-3/</id>
    <link href="https://blog.malu.tech/python-django-sql-pandas-pyecharts-data-analysis-platform-3/"/>
    <published>2024-01-14T16:00:00.000Z</published>
    <summary>
      <![CDATA[Python Django+SQL+Pandas+Pyecharts自建在线数据分析平台系列文章第三篇：页面布局&Django模板]]>
    </summary>
    <title>Python Django+SQL+Pandas+Pyecharts自建在线数据分析平台（三）</title>
    <updated>2024-01-14T16:00:00.000Z</updated>
  </entry>
  <entry>
    <author>
      <name>Road</name>
    </author>
    <category term="Python" scheme="https://blog.malu.tech/categories/Python/"/>
    <category term="Django" scheme="https://blog.malu.tech/categories/Python/Django/"/>
    <category term="数据分析" scheme="https://blog.malu.tech/categories/Python/Django/%E6%95%B0%E6%8D%AE%E5%88%86%E6%9E%90/"/>
    <category term="Python" scheme="https://blog.malu.tech/tags/Python/"/>
    <category term="Django" scheme="https://blog.malu.tech/tags/Django/"/>
    <category term="SQL" scheme="https://blog.malu.tech/tags/SQL/"/>
    <category term="Pandas" scheme="https://blog.malu.tech/tags/Pandas/"/>
    <category term="Pyecharts" scheme="https://blog.malu.tech/tags/Pyecharts/"/>
    <category term="数据分析平台" scheme="https://blog.malu.tech/tags/%E6%95%B0%E6%8D%AE%E5%88%86%E6%9E%90%E5%B9%B3%E5%8F%B0/"/>
    <content>
      <![CDATA[<blockquote><p><strong>文章来源</strong>：<a href="https://zhuanlan.zhihu.com/p/142490087">Python Django+SQL+Pandas+Pyecharts自建在线数据分析平台（一）</a><br><strong>作者</strong>：<a href="https://www.zhihu.com/people/chen-cheng-76-40">ccpic</a><br><strong>感谢</strong>：感谢作者 ccpic 分享的优质内容，本网页主要用于学习知识的存档备份，欢迎点击原网页支持作者。</p></blockquote><p>本篇是系列文章的第二篇：</p><p>（一）需求分析&amp;技术实现</p><p>（三）页面布局&amp;Django模板</p><p>（四）SQL+Pandas初步处理数据</p><p>（五）前端表单交互</p><p>（六）Ajax异步传参与加载</p><p>（七）前端数据格式的处理</p><p>（八）DataTables接管前端表格</p><p>（九）Pyecharts实现交互图表</p><p>（十）静态图表的展示</p><p>（十一）“导出数据至Excel”功能</p><p>（十二）添加和配置缓存</p><p>（十三）用户登录系统</p><p>（十四）部署Django至生产环境</p><p>既然强调是在线数据分析，那么顾名思义Web框架是一切的基础。其实很多语言都有出色的Web框架，我选用Python实现也仅仅是因为自己是二手程序员一手数据分析师，更熟悉Python罢了。而Django也不是Python下的唯一选择，Python下的还有Flask, Tornado, FastAPI等等选择，但我在实际使用中感觉基本上大同小异，使用者可以比较容易做到一通百通。</p><p>Django采用MTV（M-Model, T-Template, V-View）框架，其实是一种非常经典的Web开发结构MVC模式的变种。下面这张图很好的解释了MTV分别是什么：</p><ul><li><strong>M - Model模型，负责数据储存在的服务器端基本结构，以及数据别名、验证信息等一些基础内容。</strong><br><img src="/images/python-djangosqlpand/v2-a43fb1d914479e5b6c39aeedcc9f2844_1440w.webp"></li><li><strong>T - Template模板，负责后端传来的数据如何在Web前端展现以及前端的交互操作。</strong></li><li><strong>V - View视图，既负责与数据库做沟通增删改查，也负责加工数据决定什么数据传到前端。因此视图也是模型和模板的桥梁。</strong></li></ul><p>实际操作中，可以用cmd启动一个全新的Django项目，假设我们命名项目为datasite：</p><figure class="highlight text"><table><tr><td class="code"><pre><span class="line">django-admin startproject datasite</span><br></pre></td></tr></table></figure><p>并且在该项目下马上创建一个app如chpa_data：</p><figure class="highlight text"><table><tr><td class="code"><pre><span class="line">python manage.py startapp chpa_data</span><br></pre></td></tr></table></figure><p>也可以在Pycharm的New Project菜单中创建Django项目一歩搞定：<br><img src="/images/python-djangosqlpand/v2-27d396e45cab7a84eae8d70d9325a518_1440w.webp"></p><p>可以在项目文件夹中执行下方命令启动开发服务器：</p><figure class="highlight text"><table><tr><td class="code"><pre><span class="line">python manage.py runserver 0.0.0.0:8088</span><br></pre></td></tr></table></figure><p>在浏览器中输入 127.0.0.1:8088，出现下方页面则表示启动成功<br><img src="/images/python-djangosqlpand/v2-bef7cdc30fab0bfb8a6abad39a590fb1_1440w.webp"></p><p>此外，记得别忘了在datasite文件夹的settings.py文件内将chpa_data加入installed app：</p><figure class="highlight django"><table><tr><td class="code"><pre><span class="line"><span class="language-xml">INSTALLED_APPS = [</span></span><br><span class="line"><span class="language-xml">    ...</span></span><br><span class="line"><span class="language-xml">    &#x27;chpa_data&#x27;</span></span><br><span class="line"><span class="language-xml">]</span></span><br></pre></td></tr></table></figure><p>而此时的项目文件夹结构为：<br><img src="/images/python-djangosqlpand/v2-9cd8e286c9f9a57e268021efe9406aed_1440w.webp"></p><p>传统的Django项目第一步往往从models.py定义数据结构开始，继而编写views.py处理数据。此时项目页面还没有Template部分，一般需要手动在chpa_data文件夹下创建templates文件夹并编写对应views渲染对象的html模板。</p><p>但如果我们再次回忆文章前面部分的Django架构图片，可以意识到，<strong>Django最核心的部分其实是View层，而Model层和Template层都不是必须的。Django可以不自己编写Model，不使用自带的ORM，在View层用SQL Alchemy或Pyodbc等中间库直接用SQL语句操作数据库。Django也可以不渲染页面，在View层直接返回API的JsonResponse。</strong></p><p>在本例当中，我决定不使用Model层。个人觉得Django ORM只有在处理事务型数据时有一些易用性和可读性方面的优势，在处理大量量化数据时，可以直接编写SQL语句或用SQL语句简单Select数据后用Pandas进行复杂处理。</p><p>Dummy测试数据链接：<a href="https://share.weiyun.com/q1EZl8lW">https://share.weiyun.com/q1EZl8lW</a> 密码：xaw3kg</p><p>**我们可以在views.py用类似下面的语句实现前后端通信，这也是Django view层的基本写法。方法的唯一参数request读取前端发送的请求，再通过返回语句以期望的形式返回到前端。**这个过程中我们根据上方的想法从后端直接操作数据库将数据读取到Pandas的df，省略了Model层的操作：</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">from</span> django.http <span class="keyword">import</span> HttpResponse</span><br><span class="line"><span class="keyword">from</span> sqlalchemy <span class="keyword">import</span> create_engine</span><br><span class="line"><span class="keyword">import</span> pandas <span class="keyword">as</span> pd</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">ENGINE = create_engine(<span class="string">&#x27;mssql+pymssql://(local)/CHPA_1806&#x27;</span>) <span class="comment">#创建数据库连接引擎</span></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="keyword">def</span> <span class="title function_">index</span>(<span class="params">request</span>):</span><br><span class="line">    sql = <span class="string">&quot;Select count(*) from data&quot;</span> <span class="comment">#标准sql语句，此处为测试返回数据库data表的数据条目n，之后可以用python处理字符串的方式动态扩展</span></span><br><span class="line">    df = pd.read_sql_query(sql, ENGINE) <span class="comment">#将sql语句结果读取至Pandas Dataframe</span></span><br><span class="line">    <span class="keyword">return</span> HttpResponse(df.to_html()) <span class="comment">#渲染，这里暂时渲染为最简单的HttpResponse，以后可以扩展</span></span><br></pre></td></tr></table></figure><p>注意上方代码中sqlalchemy的数据库连接引擎写法很多变，为各式数据库Dialect结合Driver的组合，详细请参考下方的页面。此处根据你选择的数据库也有可能需要下载额外的包。</p><p>此时，如果想要在开发服务器测试上方的views，我们还需要设置一下URL。</p><p>我们选择先在项目目录datasite文件夹的urls.py里引用app chpa_data的urls：</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">from</span> django.contrib <span class="keyword">import</span> admin</span><br><span class="line"><span class="keyword">from</span> django.urls <span class="keyword">import</span> include,path</span><br><span class="line"></span><br><span class="line">urlpatterns = [</span><br><span class="line"></span><br><span class="line">    path(<span class="string">&#x27;chpa/&#x27;</span>, include(<span class="string">&#x27;chpa_data.urls&#x27;</span>)),</span><br><span class="line">    path(<span class="string">&#x27;admin/&#x27;</span>, admin.site.urls),</span><br><span class="line">]</span><br></pre></td></tr></table></figure><p>而在chpa_data目录里再手动创建urls.py，里面写入app下views.py里每个view对应的url：</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">from</span> django.urls <span class="keyword">import</span> path</span><br><span class="line"><span class="keyword">from</span> . <span class="keyword">import</span> views</span><br><span class="line"></span><br><span class="line">app_name = <span class="string">&#x27;chpa&#x27;</span>  <span class="comment"># 这句是必须的，和之后所有的URL语句有关</span></span><br><span class="line">urlpatterns = [</span><br><span class="line">    path(<span class="string">r&#x27;index&#x27;</span>, views.index, name=<span class="string">&#x27;index&#x27;</span>),</span><br><span class="line">]</span><br></pre></td></tr></table></figure><p>注意上方URL是一个嵌套的关系，也就是我要访问views.index实际上需要访问的URL的是chpa&#x2F;index而不是index。<br><img src="/images/python-djangosqlpand/v2-4985150d869f3b3fabfd058b8b06dd9f_1440w.webp"></p><p>现在的项目结构，请注意两个文件夹里都有一个urls.py, datasite里的是启动项目自动生成的，chpa_data里的是自己创建的。前者引用后者。</p><p>在浏览器输入127.0.0.1:8088&#x2F;chpa&#x2F;index，出现下方页面，说明测试成功，返回了chpa_data app内views.py里”Select count(*) from data”的结果6065706，而0是Pandas的行索引。<br><img src="/images/python-djangosqlpand/v2-60646fb4a9bc6246e2d50403ccbb2005_1440w.webp"></p><p>至此，我们打通了这个项目从后端到前端的数据链，实现了从浏览器的一行URL返回指定数据库SQL查询结果的过程，看似简单，却是最关键的一个步骤。</p><p>下一篇请移步：</p>]]>
    </content>
    <id>https://blog.malu.tech/python-django-sql-pandas-pyecharts-data-analysis-platform-2/</id>
    <link href="https://blog.malu.tech/python-django-sql-pandas-pyecharts-data-analysis-platform-2/"/>
    <published>2024-01-07T16:00:00.000Z</published>
    <summary>Python Django+SQL+Pandas+Pyecharts自建在线数据分析平台系列文章第二篇：初步搭建Django环境</summary>
    <title>Python Django+SQL+Pandas+Pyecharts自建在线数据分析平台（二）</title>
    <updated>2024-01-07T16:00:00.000Z</updated>
  </entry>
  <entry>
    <author>
      <name>Road</name>
    </author>
    <category term="mini主机" scheme="https://blog.malu.tech/categories/mini%E4%B8%BB%E6%9C%BA/"/>
    <category term="装机" scheme="https://blog.malu.tech/tags/%E8%A3%85%E6%9C%BA/"/>
    <category term="迷你主机" scheme="https://blog.malu.tech/tags/%E8%BF%B7%E4%BD%A0%E4%B8%BB%E6%9C%BA/"/>
    <category term="Opencore" scheme="https://blog.malu.tech/tags/Opencore/"/>
    <category term="Hackintosh" scheme="https://blog.malu.tech/tags/Hackintosh/"/>
    <category term="1L小主机" scheme="https://blog.malu.tech/tags/1L%E5%B0%8F%E4%B8%BB%E6%9C%BA/"/>
    <category term="8100B" scheme="https://blog.malu.tech/tags/8100B/"/>
    <category term="M910X" scheme="https://blog.malu.tech/tags/M910X/"/>
    <content>
      <![CDATA[<p>省流助手，EFI下载链接：</p><p><a href="https://github.com/Road-tech/Hackintosh_LenovoM910X_8100B_RX460_OC">https://github.com/Road-tech/Hackintosh_LenovoM910X_8100B_RX460_OC</a></p><h1 id="关于M910X"><a href="#关于M910X" class="headerlink" title="关于M910X"></a>关于M910X</h1><p>联想的M910X（p320 tiny），一个非常好玩的1L迷你小主机。Q270的主板，双M.2插槽、一个PCIe扩展槽、双通道ddr4、6个USB，同时是最后一代可以刷bios上魔改U的联想小主机。再往上的M920x，P340都是双BIOS设计，无法刷bios了，也基本告别了便宜好玩的ES版CPU或者魔改U。</p><p>现在这台小主机性价比非常高，700出头的价格就买到这样的强悍扩展性，放在这个价位简直无敌的存在，而且可玩性非常高。这个价格换成300系芯片组的小主机，基本都没有双M.2接口（除了dell 7080mff 低压版），更别说PCIe扩展了。而他的下一代M920X，现在还要1300的价格，相比起来只多了个typc-C接口，不过原生可以上8代U，但只能支持正式版。M910x原配的显卡为RX460，现在咸鱼原厂全新只要600左右的价格。而M920x配套的rx560现在咸鱼要差不多1000，一个性能差不多的马甲卡居然贵那么多。当然如果不追求黑苹果，只为最强的独显性能，最新的M930X，原厂可以选配到P620。当然动手能力强的可以上GTX1650，妥妥的小钢炮，就是要切挡板，考验手艺。</p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2022/01/24-10-58-47-lenovo-thinkCentre-M910x-tiny-hero-front-1.webp"></p><p>图源联想官网<a href="https://www.lenovo.com/us/en/p/desktops/thinkcentre/m-series-tiny/m910x-tiny/11tc1mt910x">https://www.lenovo.com/us/en/p/desktops/thinkcentre/m-series-tiny/m910x-tiny/11tc1mt910x</a></p><h2 id="我的硬件配置"><a href="#我的硬件配置" class="headerlink" title="我的硬件配置"></a>我的硬件配置</h2><table><thead><tr><th></th><th align="center">Specifications &#x2F; 型号</th><th align="center">Note &#x2F; 备注</th></tr></thead><tbody><tr><td>Motherboard&#x2F;主板:</td><td align="center">Lenovo M910X Q270</td><td align="center">1L 迷你主机</td></tr><tr><td>CPU&#x2F;处理器:</td><td align="center">I3-8100B</td><td align="center">闪电家魔改U</td></tr><tr><td>CPU Cooler&#x2F;散热器:</td><td align="center">准系统自带</td><td align="center"></td></tr><tr><td>Hard Drive&#x2F;硬盘:</td><td align="center">Hikvision C2000pro 512gb</td><td align="center"></td></tr><tr><td>RAM&#x2F;内存:</td><td align="center">Samsung 8G DDR4 2666MHz X1</td><td align="center"></td></tr><tr><td>Wireless Card&#x2F;无线网卡:</td><td align="center">BCM94360cs2+转接卡</td><td align="center">白果拆机卡</td></tr><tr><td>Tower Case&#x2F;机箱:</td><td align="center">准系统自带</td><td align="center"></td></tr><tr><td>Power&#x2F;电源:</td><td align="center">Lenovo 20v 6.75A 135w DC adapter</td><td align="center"></td></tr></tbody></table><hr><h1 id="一些折腾点"><a href="#一些折腾点" class="headerlink" title="一些折腾点"></a>一些折腾点</h1><h2 id="关于魔改U"><a href="#关于魔改U" class="headerlink" title="关于魔改U"></a>关于魔改U</h2><p>聊回这台M910X，要上这个魔改U8100B&#x2F;8500B&#x2F;8700B，需要刷入修好的BIOS。一般魔改u的老板都会提供一个修好的BIOS，而闪电家给我提供的bios，虽然能点亮，但是因为没有写入S&#x2F;N等信息，开机滴滴滴两声报错，而且BIOS版本也太老了，还关不掉cfg-lock。</p><p>于是需要自己修改bios，如果你有6代或者7代的亮机U，这是最方便的，直接进windows更新bios到最新版本。这里感叹一下，这台2018年就发布的机器，到2021-7-6居然还更新了一版BIOS，感觉换成那些零售的diy主板，早就停止支持了。</p><p>官方BIOS下载地址在这里：<a href="https://newsupport.lenovo.com.cn/driveList.html?fromsource=driveList&selname=ThinkCentre%20M910x">点我下载</a></p><p>更新完BIOS后，用编程器把BIOS提取出来，使用D大的工具进行魔改，具体操作请参考：<a href="http://www.smxdiy.com/thread-1299-1-1.html">部分 Lenovo 联想 LGA1151 主机 支持 8 代 9 代 BIOS 修改工具</a></p><p>如果没有亮机U，那只能用编程器直接把BIOS提取出来，参照上面链接里的强刷教程，刷入魔改bios后，进Windows用WriteSN工具补回S&#x2F;N等信息。</p><p>闪电家提供的BIOS和自己修改的BIOS我都放在了<a href="https://github.com/Road-tech/Hackintosh_LenovoM910X_8100B_RX460_OC/tree/main/%E9%AD%94%E6%94%B9BIOS">魔改BIOS</a>的文件夹内，可自行下载研究。</p><p><em><strong>如果你选择直接刷入这两版BIOS，而不是自己提取修改，请务必用WriteSN工具补回原机的S&#x2F;N等信息</strong></em></p><p>BIOS芯片为25L12873F，具体位置参考这个图（图源自SMZDM的<a href="https://zhiyou.smzdm.com/member/9509386572/">折了个腾</a>）</p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2022/01/24-00-47-21-5ecdd94a3f0322866.jpg_e1080.webp" alt="BIOS芯片"></p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2022/01/24-00-47-11-5ecdd94a44bfd6611.jpg_e1080.webp" alt="具体位置"></p><h2 id="关于CFG-Lock"><a href="#关于CFG-Lock" class="headerlink" title="关于CFG-Lock"></a>关于CFG-Lock</h2><p>一般来说，黑苹果想要实现完美的休眠，关闭CFG-Lock是必要条件的。</p><p>如果你选择刷入我自己改好的BIOS，那这个BIOS已经把隐藏的CFG-lock开关显示出来，直接在BIOS里面关闭就好了。</p><p>如果选择自己修改BIOS，BIOS没有CFG-lock选项，可以用opencore引导解锁。在启动菜单选择页面选择ControlMsrE2，我已经在EFI默认配置了unlock参数，进入后可以看到CFG-lock的状态，同时尝试解锁CFG-lock。</p><h2 id="关于显卡"><a href="#关于显卡" class="headerlink" title="关于显卡"></a>关于显卡</h2><p>这台小主机有两张显卡，分别是CPU的UHD630以及独显RX460。</p><p>在macOS里，即便有独显，核显还是有作用的，可以用于加速，所以630核显直接配置<code>AAPL,ig-platform-id</code>为<code>0300913E </code>，不需要做更多的显卡输出修复。</p><p>独显直接免驱动，Emmm，这算是我折腾过最简单的方案了。</p><h2 id="关于声卡仿冒"><a href="#关于声卡仿冒" class="headerlink" title="关于声卡仿冒"></a>关于声卡仿冒</h2><p>省流助手：<code>layout-id</code>为<code>12</code>，也就是<code>0C000000</code></p><p>一开始我参照了网上现有的opencore配置，发现声卡仿冒的<code>layout-id</code>一般都是设置为11和21两种。我分别试了下，设置11的时候主机的内置音箱有声音，插耳机没声音。设置为21的时候情况相反。不完美很难受</p><p>后来网上查资料看到这种情况，需要自己定制仿冒声卡，于是我参考了<a href="https://shuiyunxc.gitee.io/2020/03/21/M920x/index/">OpenCore引导安装联想-M920x黑苹果之历程</a>这篇文章，按照文章给出的参数自己编译了AppleALC.Kext。但是怎么弄都不行，明明所有参数都是对的，最后才发现原来这是M920X教程，汗- 。-！。（M910X的兄弟型号是P320 tiny，导致我老是把M910X记成M920X）</p><p>M920X的声卡是ALC235，而M910X的声卡是ALC294，也就是这些参数并不通用。自己仿冒声卡步骤超级无敌复杂，无敌头疼。但是在GitHub翻AppleALC的代码的时候，发现2018年7月的时候MacPeet提交了关于Realtek ALC294 for Lenovo M710Q的仿冒配置，<code>layout-id</code>为12。考虑到同一代的小主机的硬件设计高度相同，于是就去试了下12，果然是完美的！内置音箱和耳机都正常工作。所以<code>layout-id</code>设置为12就好了！感谢MacPeet大佬。</p><h2 id="关于网卡的选择"><a href="#关于网卡的选择" class="headerlink" title="关于网卡的选择"></a>关于网卡的选择</h2><p>黑苹果的网卡选择有很多，图简单省事的话，可以选黑果小兵的BCM94360Z3或者BCM94360Z4。应该加个kext就可以完美驱动了。<br>链接可以参考这<a href="https://blog.daliansky.net/BCM94360Z4-m.2-NGFF-interface-four-antenna-notebook_small-host-dedicated-black-Apple-wireless-network-card-driver-tutorial.html">【黑果小兵独家】BCM94360Z4&#x2F;BCM94360Z3 m.2 NGFF接口四天线笔记本&#x2F;小主机专用黑苹果无线网卡驱动教程</a></p><p>不过考虑到m910x内部对无线网卡的高度没什么限制，最推荐的还是苹果iMac拆机的BCM94360cs2或者BCM943602cs配合转接卡，什么驱动都不用补，最省事。但是长度有限制，BCM94360cs2要磨掉一点PCB才能刚刚好放进去，更长的BCM943602cs就别想了。所以这里推荐反向的转接卡，再用点热熔胶固定。</p><p>具体可以参考这个图：（图源自SMZDM的<a href="https://zhiyou.smzdm.com/member/9509386572/">折了个腾</a>）</p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2022/01/24-00-47-41-5ecdd94fe51db2827.jpg_e1080.webp" alt="反向转接卡"></p><h2 id="关于内置喇叭升级"><a href="#关于内置喇叭升级" class="headerlink" title="关于内置喇叭升级"></a>关于内置喇叭升级</h2><p>M910X内部有一个很小的喇叭（下图红色框），虽然上文说的AppleALC设定好ID后，内置喇叭和3.5mm耳机接口都可以正常工作，但是内置喇叭声音跟蚊子一样，音量调到最大也只有一点点且毫无质感。看到闲鱼上有卖据说是顶配P330 tiny上用的大功率喇叭，功率高达2W（ -.- ）。</p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2022/01/31-14-47-29-IMG_2733%20%E6%8B%B7%E8%B4%9D.webp"></p><p>不过卖家也不知道m910x能不能用。自己看了下M910X跟M920X差不多都有个螺丝孔和支架（上图篮框），接口也一样（上图绿框），决定买个试试。（但是这玩意真心贵，毫无技术含量的东西居然要80+）。</p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2022/01/31-14-48-52-IMG_2735%20%E6%8B%B7%E8%B4%9D.webp"></p><p>上图为装好后的效果，确实是完美兼容的，音量也大了很多，音量开到一半就感觉很足了。但是声音依旧没什么质感。也就是从不及格到及格的水平，可以用了，但是绝对对不起他的价格。</p><h2 id="关于BIOS设定"><a href="#关于BIOS设定" class="headerlink" title="关于BIOS设定"></a>关于BIOS设定</h2><h3 id="Disable："><a href="#Disable：" class="headerlink" title="Disable："></a>Disable：</h3><ul><li><p>设备：</p></li><li><p>System Agent(SA) Configuration -&gt; VT-d </p></li><li><p>ATA设备清单 -&gt; Configure SATA as -&gt; AHCI  </p></li><li><p>显示菜单 -&gt; Auto</p></li><li><p>网络菜单 -&gt; PXE启动 </p></li><li><p>高级菜单：</p></li><li><p>CPU Configuration -&gt; SW Guard Extensions (SGX)</p></li><li><p>Power &amp; Performance -&gt; CPU -&gt; CPU Lock Configuration -&gt; CFG Lock  </p></li><li><p>启动菜单：</p></li><li><p>兼容模块</p></li></ul><h3 id="Enable："><a href="#Enable：" class="headerlink" title="Enable："></a>Enable：</h3><ul><li><p>设备：</p></li><li><p>System Agent(SA) Configuration -&gt; Above 4G MMIO BIOS assignment  </p></li><li><p>高级菜单： </p></li><li><p>CPU Configuration -&gt; Intel (VMX) Virtualization Technology (VT-x)  </p></li><li><p>启动菜单：</p></li><li><p>启动方式：UEFI</p></li></ul><hr><h1 id="Functions-功能"><a href="#Functions-功能" class="headerlink" title="Functions&#x2F;功能"></a>Functions&#x2F;功能</h1><h3 id="Work："><a href="#Work：" class="headerlink" title="Work："></a>Work：</h3><ul><li>All DP ports (1080p) on RX460  </li><li>Audio output on DP  </li><li>All USB ports  </li><li>Wi-Fi &amp; Bluetooth  </li><li>3.5mm Audio Jack and Internal Mic</li><li>Airdrop  </li><li>AirPlay  </li><li>Continuity  </li><li>QE&#x2F;CI of Intel UHD 630 &amp; rx460</li><li>CPU Power Management</li><li>Sleep</li></ul><h3 id="Not-working"><a href="#Not-working" class="headerlink" title="Not working:"></a>Not working:</h3><ul><li><ul><li></li></ul></li></ul><h3 id="Not-tested-yet"><a href="#Not-tested-yet" class="headerlink" title="Not tested yet:"></a>Not tested yet:</h3><ul><li>4k display</li></ul><hr><h1 id="Performance-展示"><a href="#Performance-展示" class="headerlink" title="Performance&#x2F;展示"></a>Performance&#x2F;展示</h1><p>我已经超级无敌懒，根本不想自己拍照，都是网上现找的图，如侵删。</p><p>以下图源自<a href="https://forums.lenovo.com/t5/ThinkCentre-A-E-M-S-Series/Lenovo-M910x-Tiny-Extreme-RX-460-graphics-option-details-and-avail/m-p/3725943?page=2">English Community-Lenovo Community</a>以及<a href="https://www.lenovo.com/us/en/p/desktops/thinkcentre/m-series-tiny/m910x-tiny/11tc1mt910x">联想官网</a></p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2022/01/24-11-10-50-lenovo-thinkCentre-M910x-tiny-hero.webp"></p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2022/01/24-11-12-54-lenovo-thinkCentre-M910x-tiny-mdp-ports-5.webp"></p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2022/01/24-11-12-24-lenovo-thinkCentre-M910x-tiny-left-right-side-7.webp"></p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2022/01/24-11-13-35-121854iF16DE5EBBC821E4B.webp"></p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2022/01/24-11-13-41-121853iE5F561AB0C4920BB.webp"></p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2022/01/24-10-33-45-CPU%26%E6%98%BE%E5%8D%A1.webp" alt="CPU变频&amp;显卡驱动正常"></p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2022/01/24-10-33-54-USB%E5%AE%9A%E5%88%B6.webp" alt="USB定制"></p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2022/01/24-10-34-02-%E8%93%9D%E7%89%99.webp" alt="蓝牙工作正常"></p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2022/01/24-10-34-12-Wi-Fi.webp" alt="Wi-Fi工作正常"></p><hr><h1 id="Reference-参考"><a href="#Reference-参考" class="headerlink" title="Reference&#x2F;参考"></a>Reference&#x2F;参考</h1><p><a href="https://dortania.github.io/OpenCore-Install-Guide/">Dortania’s OpenCore Install Guide</a></p><p><a href="https://blog.daliansky.net/OpenCore-BootLoader.html">精解OpenCore</a> - <a href="https://blog.daliansky.net/">黑果小兵的部落阁 </a></p><p><a href="https://blog.xjn819.com/?p=543">使用OpenCore引导黑苹果</a> - <a href="https://blog.xjn819.com/">XJN</a> </p><p><a href="https://github.com/acidanthera/AppleALC">acidanthera&#x2F;AppleALC</a></p><p><a href="https://github.com/ylen0l/Hackintosh-Lenovo-Thinkcentre-M910x-OpenCore-Efi">xia54&#x2F;Hackintosh-Lenovo-Thinkcentre-M910x-OpenCore-Efi</a></p><p><a href="https://osxlatitude.com/forums/topic/13992-success-lenovo-m720q-m920q-p330-tiny-catalina-10156-opencore/">[SUCCESS] Lenovo M720q , M920q, P330 Tiny Catalina 10.15.6 OPENCORE</a></p><p><a href="https://github.com/chencaidy/Hackintosh-OC-Lenovo-ThinkCentre-M920x">chencaidy&#x2F;Hackintosh-OC-Lenovo-ThinkCentre-M920x</a></p><p><a href="https://post.smzdm.com/p/ammkv6vv/">一台比较完美的黑苹果小主机 联想M910Q折腾记 opencore EFI分享</a></p>]]>
    </content>
    <id>https://blog.malu.tech/Hackintosh_LenovoM910X_8100B_RX460_OC/</id>
    <link href="https://blog.malu.tech/Hackintosh_LenovoM910X_8100B_RX460_OC/"/>
    <published>2022-01-23T16:00:00.000Z</published>
    <summary>联想的M910X（p320 tiny），一个非常好玩的1L迷你小主机。Q270的主板，双M.2插槽、一个PCIe扩展槽、双通道ddr4、6个USB，同时是最后一代可以刷bios上魔改U的联想小主机。再往上的M920x，P340都是双BIOS设计，无法刷bios了，也基本告别了便宜好玩的ES版CPU或者魔改U。...</summary>
    <title>1L小主机系列 - Lenovo M910X</title>
    <updated>2022-01-30T16:00:00.000Z</updated>
  </entry>
  <entry>
    <author>
      <name>Road</name>
    </author>
    <category term="软件技巧" scheme="https://blog.malu.tech/categories/%E8%BD%AF%E4%BB%B6%E6%8A%80%E5%B7%A7/"/>
    <category term="配置记录" scheme="https://blog.malu.tech/tags/%E9%85%8D%E7%BD%AE%E8%AE%B0%E5%BD%95/"/>
    <category term="OpenWrt" scheme="https://blog.malu.tech/tags/OpenWrt/"/>
    <content>
      <![CDATA[<p>最近收了一台DIY属性拉满的迷你4G无线路由，出自大R杂货的MagicBox双频路由。大佬定制的主板，小巧的体积加上支持双频WiFi、LTE&#x2F;4G网络、OpenWrt等属性，可以说极客感满满。</p><p>不过大佬默认只提供原版纯净的固件，什么功能都没有，甚至主题都没有。虽然也没法集成太多的功能，但是后续想要更新版本就不方便了，还需要自己重新安装各种功能和配置各种参数。所以还是研究了下GitHub action和Openwrt，把自己需要功能和配置编辑好。后续openwrt只要更新了，GitHub便会自动帮我编译新固件。所以这篇文章就是记录各种配置的过程，方便日后的查询。</p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/29-11-14-55-IMG_2409.webp"></p><h2 id="硬件配置："><a href="#硬件配置：" class="headerlink" title="硬件配置："></a>硬件配置：</h2><ul><li><p>高通 QCA9531 550Mhz CPU + 9887 5G Wi-Fi 芯片</p></li><li><p>16M闪存 &#x2F; 128M内存</p></li><li><p>433Mbps + 300Mbps 双频 Wi-Fi</p></li><li><p>USB 扩展口（ LTE 版两个 &#x2F; Wi-Fi 版一个）</p></li><li><p>两个百兆网口（默认 1WAN 1LAN）</p></li><li><p>TF 卡槽（Wi-Fi 版无）</p></li><li><p>LTE 版为 Type-C 供电 &#x2F; Wi-Fi 版为 Micro USB 供电。</p></li><li><p>4G LTE 版为下图3D打印的黑色尼龙外壳 &#x2F; Wi-Fi 版为上图亚克力外壳</p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/29-11-14-45-IMG_2410.webp"></p></li></ul><h2 id="Openwrt相关"><a href="#Openwrt相关" class="headerlink" title="Openwrt相关"></a>Openwrt相关</h2><h3 id="选择openwrt版本"><a href="#选择openwrt版本" class="headerlink" title="选择openwrt版本"></a>选择openwrt版本</h3><p>首先openwrt分为稳定版和开发版，目前稳定版的版本定在19.07，而开发版为20.XX。目前openwrt对9531的支持直到19.07稳定版，目前开发版是不支持的。</p><p>然后openwrt也会有不同的分支，除了官方原版<a href="https://github.com/openwrt/openwrt">openwrt</a>，还有<a href="https://github.com/coolsnowwolf">coolsnowwolf</a>大佬的<a href="https://github.com/coolsnowwolf/lede">LEDE</a>、<a href="https://github.com/Lienol">Lienol</a>大佬的<a href="https://github.com/Lienol/openwrt">openwrt</a>。</p><p>我很浅略的对比了三个版本的代码，LEDE和Lienol的openwrt会针对我们的使用习惯进行优化，比如默认生成Wi-Fi的ssid名称，会区分2.4G和5G。同时会集成更多的功能，让我们的编译更加方便，不用一个个的去寻找添加，而且自带的源也能保证与系统稳定运行。当然估计还有一些性能上的优化，这方面我就看不出来了。</p><p>因为AR9531目前只能支持19.07，所以在编译的时候需要留意选择正确的版本，官方原版的openwrt和Lienol大的不同版本是在GitHub上的不同分支，而coolsnowwolf大佬的不同版本是不同的库，这里是要留意的。</p><p><a href="https://github.com/coolsnowwolf">coolsnowwolf</a>大佬的lede不同版本的库:</p><ul><li><p>稳定版   <a href="https://github.com/coolsnowwolf/openwrt">https://github.com/coolsnowwolf/openwrt</a></p></li><li><p>开发版   <a href="https://github.com/coolsnowwolf/lede">https://github.com/coolsnowwolf/lede</a></p></li></ul><h3 id="生成编译配置"><a href="#生成编译配置" class="headerlink" title="生成编译配置"></a>生成编译配置</h3><p>这一步目前没有其他方法，还是需要自行搭建一个ubuntu平台，拉取整个openwrt的库再进行<code>make menuconfig</code> 操作生成编译配置。不过好在这个配置确定后，后续不需要修改了。所以也就麻烦这一次。</p><h4 id="集成LTE所需驱动"><a href="#集成LTE所需驱动" class="headerlink" title="集成LTE所需驱动"></a>集成LTE所需驱动</h4><p>依照老板大R的要求，驱动LTE所需的3个驱动，分别是：</p><p>kmod-usb-net</p><p>kmod-usb-net -&gt; kmod-usb-net-rndis</p><p>usb-modeswitch</p><h4 id="添加所需的功能"><a href="#添加所需的功能" class="headerlink" title="添加所需的功能"></a>添加所需的功能</h4><p>这里可以按需添加，功能主要都集成在Luci下。Lede和Lienol的版本集成的功能会明显更多更方便。  </p><p>需要注意的是，添加功能后要注意固件的大小，不要让最终的固件大于16M导致编译失败。同时如果出现功能太多需要调整配置，我强烈建议先删除默认的.config文件重新配置。因为你选择一个功能时，可能会自动选择所需的各种以来，而你只是单纯的取消这个功能，相关的依赖并不会联同取消，这样很有可能会出现你明明取消了这个功能，但是固件并没有减少，因为相关的大量依赖还是被编译进去了。  </p><p>下面记录一些相关的命令：</p><ol><li><p>添加luci-theme-argon主题</p><figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">cd openwrt/package</span><br><span class="line">git clone https://github.com/jerrykuku/luci-theme-argon.git #拉取主题 </span><br><span class="line">sed -i &#x27;s/luci-theme-bootstrap/luci-theme-argon/g&#x27; feeds/luci/collections/luci/Makefile #修改默认的主题</span><br><span class="line">make menuconfig #选择 LUCI-&gt;Theme-&gt;Luci-theme-argon  </span><br></pre></td></tr></table></figure></li><li><p>中文支持</p><figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">make menuconfig #选择LuCI-&gt;Modules-&gt;Translations-&gt;Chinese</span><br></pre></td></tr></table></figure></li></ol><h4 id="修改默认的设定"><a href="#修改默认的设定" class="headerlink" title="修改默认的设定"></a>修改默认的设定</h4><ol><li><p>默认开启wifi </p><figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">sed -i &#x27;s/disabled=1/disabled=0/g&#x27; package/kernel/mac80211/files/lib/wifi/mac80211.sh</span><br><span class="line">sed -i &#x27;s/OpenWrt/Road-MagicBox/g&#x27; package/kernel/mac80211/files/lib/wifi/mac80211.sh</span><br></pre></td></tr></table></figure></li><li><p>修改路由默认ip</p><figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">sed -i &#x27;s/192.168.1.1/192.168.8.1/g&#x27; package/base-files/files/bin/config_generate</span><br><span class="line"><span class="meta prompt_">#</span><span class="language-bash">把lan口默认ip由192.168.1.1改成192.168.8.1</span></span><br></pre></td></tr></table></figure></li><li><p>添加wwan接口</p><figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">sed -i &#x27;$d&#x27; package/base-files/files/bin/config_generate</span><br><span class="line">sed -i &#x27;$a uci set network.wwan=interface&#x27; package/base-files/files/bin/config_generate</span><br><span class="line">sed -i &#x27;$a uci set network.wwan.ifname=eth2&#x27; package/base-files/files/bin/config_generate</span><br><span class="line">sed -i &#x27;$a uci set network.wwan.proto=dhcp&#x27; package/base-files/files/bin/config_generate</span><br><span class="line">sed -i &#x27;$a uci set network.wwan.up=1&#x27; package/base-files/files/bin/config_generate</span><br><span class="line">sed -i &#x27;$a uci commit&#x27; package/base-files/files/bin/config_generate</span><br></pre></td></tr></table></figure></li><li><p>wwan接口添加防火墙</p><figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">sed -i &quot;19a \ \ \ \ \ \ \ \ list   network          &#x27;wwan&#x27; &quot; package/network/config/firewall/files/firewall.config</span><br></pre></td></tr></table></figure></li><li><p>修改主机名</p><figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">sed -i &#x27;s/OpenWrt/MagicBox/g&#x27; package/base-files/files/bin/config_generate</span><br></pre></td></tr></table></figure></li><li><p>设定root密码为password</p><figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">sed -i &#x27;1d&#x27; package/base-files/files/etc/shadow</span><br><span class="line">sed -i &#x27;1i root:$1$H\/ab6bvd$yWkIzUrKuLPTNHY9akBDC0:18988:0:99999:7:::&#x27;  package/base-files/files/etc/shadow</span><br></pre></td></tr></table></figure></li></ol><h2 id="Github-action相关"><a href="#Github-action相关" class="headerlink" title="Github action相关"></a>Github action相关</h2><p>自动编译脚本源自<a href="https://github.com/P3TERX">P3TERX</a>&#x2F;<a href="https://github.com/P3TERX/Actions-OpenWrt">Actions-OpenWrt</a>，脚本使用说明：<a href="https://github.com/P3TERX/Actions-OpenWrt">English</a> | <a href="https://p3terx.com/archives/build-openwrt-with-github-actions.html">中文</a></p>]]>
    </content>
    <id>https://blog.malu.tech/OpenwrtConfigurationRecord/</id>
    <link href="https://blog.malu.tech/OpenwrtConfigurationRecord/"/>
    <published>2022-01-04T16:00:00.000Z</published>
    <summary>本文记录Openwrt配置的全过程</summary>
    <title>OpenWrt配置记录</title>
    <updated>2022-01-05T16:00:00.000Z</updated>
  </entry>
  <entry>
    <author>
      <name>Road</name>
    </author>
    <category term="mini主机" scheme="https://blog.malu.tech/categories/mini%E4%B8%BB%E6%9C%BA/"/>
    <category term="装机" scheme="https://blog.malu.tech/tags/%E8%A3%85%E6%9C%BA/"/>
    <category term="迷你主机" scheme="https://blog.malu.tech/tags/%E8%BF%B7%E4%BD%A0%E4%B8%BB%E6%9C%BA/"/>
    <category term="thin itx" scheme="https://blog.malu.tech/tags/thin-itx/"/>
    <category term="MK1" scheme="https://blog.malu.tech/tags/MK1/"/>
    <category term="Home Server" scheme="https://blog.malu.tech/tags/Home-Server/"/>
    <category term="3D打印" scheme="https://blog.malu.tech/tags/3D%E6%89%93%E5%8D%B0/"/>
    <content>
      <![CDATA[<h2 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h2><p>      许久前在CHH看到赵总设计的这款机箱后（<a href="https://www.chiphell.com/thread-1424686-1-1.html">链接在此</a>），就深深中毒了。奈何这片文章发表与2015年，且据闻这个机箱的产量极少，意味着想拥有一台几乎不可能了。</p><p>      去年diy了一台Mac mini，同样都是使用的Thin itx主板，虽然千辛万苦等到了成品套件，但是对于套件的散热能力并不满意。也尝试了几种方案，包括魔改下压式散热、优化风道版的尼米兹散热、加大散热鳍片的面积，最终的效果也还是不太满意。毕竟受限于风扇的尺寸，再怎么优化，没有足够的风量带走热量，这台diy的Mac mini永远是个小闷罐。</p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-38-19-Mac%20mini%20%E6%96%B0%E6%95%A3%E7%83%AD%E6%96%B9%E6%A1%88.webp" alt="魔改下压散热方案（图源B站Tifika）vs优化风道版的尼米兹散热方案（图源活久见大佬）"></p><p>​      另外市面上售卖的thin itx机箱，几乎都是洞洞流，再搭配超薄的下压式散热。丑爆之余，散热功率也就那样。</p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-39-07-%E5%B8%82%E9%9D%A2%E4%B8%8A%E7%9A%84thin%20itx%E6%9C%BA%E7%AE%B1.webp"></p><p>      So，在此想起赵总的这款MK1，4个风扇，气流覆盖整个主板。垂直风道，烟囱效应，热空气自然的往上走，享受物理学加成，理论上风扇转速不用很高可以换来不错的效果。而且这样的散热方式，可以很快乐的使用侧透面板，告别洞洞流机箱。</p><p>      之前折腾Mac mini的时候，入手了3D打印机，也自学了Sketchup，画个简单的模型也没啥问题。虽然工具都不是专业的，但也算是具备折腾的条件了。虽然说这个机箱的结构并不算复杂，但是从一开始的设计到各种细节调整，前前后后也折腾了大半年，才到现在相对满意的样子。为了避免出行日后买不到零件，于是有了尽可能用市售的通用零件的原则，实现了原版四个风扇上吹风，垂直风道的布局，并且原版有的监控小屏幕、2.5寸盘位都复现了。因为我不会画pcb，所以最难搞的就是那块风扇hub调速器，最终也在淘宝找到几乎一样的版本。</p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-39-26-fan-hub.webp"></p><p>      最后放个图，左边是原版的赵总MK1，右边是我复刻的版本。虽然质感方面完全没得比，但是一眼过去也能凑合。</p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-37-47-MK1%E5%8E%9F%E7%89%88VS%E5%9B%BE.webp" alt="MK1原版VS复刻版"></p><h2 id="机箱特性"><a href="#机箱特性" class="headerlink" title="机箱特性"></a>机箱特性</h2><ol><li><p>沿用原版四个风扇上吹风，垂直风道的布局；</p></li><li><p>通用的挡板孔位，兼容标准的thin itx 主板；</p></li><li><p>支持约3寸的LCD监控屏；</p></li><li><p>支持4个4cm风扇，具体高度如4010、4015、4020、4028尺寸的风扇都兼容；</p></li><li><p>通过风扇hub调速器实现风扇调速，支持主板pwm调速和手动调速，最高可支持5个风扇；</p></li><li><p>支持1个2.5寸硬盘，M.2硬盘支持情况取决于主板；</p></li><li><p>机箱3D打印（穷+没渠道CNC），机箱每部分的颜色可以自由定制。</p></li><li><p>成品尺寸211.7x183.2x53.8mm（高x深x宽，不含底座）。原版尺寸220x190x50mm</p></li></ol><h2 id="迭代版本介绍"><a href="#迭代版本介绍" class="headerlink" title="迭代版本介绍"></a>迭代版本介绍</h2><p>      首先1.0版本，就是照搬原版的结构，用两根长杆链接机箱前后面板和主板。当然验证后就发现了几个问题：</p><ol><li>原版的金属材质可以直接攻丝固定零件，而3D打印的部件因为强度不够，需要热熔螺母加热嵌进去，在用螺丝把两根杆链接前后面板。但是这样固定的机箱并不稳定，容易晃动。主板反而成为稳定整个机箱的部件，感觉会影响主板寿命。</li><li>还是因为3D打印的材质强度不够，只用两个杆固定主板的话，原版目测1mm厚的铝材强度就够了，而PLA材质需要把杆加厚到6mm强度才够，会导致机箱体机增加。</li><li>机箱面板的前半部分壁厚设计太小了，而且一个长条形的形状加上3D打印的材质容易冷却收缩，导致部件打印出来非常容易变弯，不管是PLA材质还是尼龙烧结都很难避免。<br><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-40-21-%E6%9C%BA%E7%AE%B1V1.0.webp"></li></ol><p>      接下来的2.0版本就重新画了主板的支架，将两根长杆换成了一个框，想着这样固定机箱应该就不会那么容易晃动了。然后想到这么一个简单的框，不需要CNC加工的话，弄成金属的也不会很贵，然后就TB加工了一个，也不贵，一个框20，但是毛刺太多，割手，也不好看。精加工嘛，价格又上去了，不适合我的钱包，所以还是考虑回打印的方案。</p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-40-31-itx%E6%94%AF%E6%9E%B6.webp"></p><p>      再下来的3.0版本呢，用回PLA打印主板支架。但是又回到了老问题，一个正方框，3D打印的材质强度还是不够，还是晃。怎么解决呢？既然正方形支架不稳定，那就改成圆角的，然后加厚到4mm。这个时候又太厚了，影响体积，于是想了个偷鸡的方法，把前后面板结合的位置减薄。最终修个圆角美化一些，再恬不知耻的打上自己的名字，装逼的同时还能节省一点点打印耗材。<br><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-41-48-itx%E6%94%AF%E6%9E%B6-%E5%9C%86%E8%A7%922.webp"></p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-41-56-itx%E6%94%AF%E6%9E%B6-%E5%9C%86%E8%A7%92-2mm-V2.0.webp"></p><p>      最终再弄个开关，画好io口和各种螺丝孔的位置，增加LCD监控屏和风扇Hub的固定位置，再调整亿点点细节机箱就弄完了。（半年就过去了，耗材都浪费了了好几卷）</p><p>      最终设计图就如下：</p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-42-04-%E6%9C%BA%E7%AE%B1-5mm-V0.webp"></p><h2 id="细节展示"><a href="#细节展示" class="headerlink" title="细节展示"></a>细节展示</h2><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-42-10-IMG_2088-1.webp"></p><p>      全览</p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-44-29-IMG_2089-1.webp"></p><p>      主板支架，预留沉头螺丝孔，M3孔径。</p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-44-38-IMG_2094-1.webp"></p><p>      分别是前后面板，面板四周预留直径4mm深度3mm的孔，用来植入热熔螺母。</p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-44-44-IMG_2122-1.webp"></p><p>      这是两种镶入铜螺母，左边的是注塑螺母，右边的是热熔螺母。可以看到两种螺母的纹路不同，这里应该使用热熔螺母才对。一开始我用了注塑螺母，导致螺母无法固定牢固。</p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-44-48-IMG_2125.webp"></p><p>      这里使用的是M3*3-4.2的热熔螺母，可以看到螺母一端有一圈比较小的头部，小头直径是4mm的，刚好可以放进预留的孔位，这样就能很精准的定位螺母的位置。 </p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-45-09-IMG_2126-1.webp"></p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-45-15-IMG_2131_2.gif"></p><p>      植入热熔螺母很简单，拔螺母放正，随便一个电烙铁，≈200度，保持垂直，轻轻压上去就可以了。压入后再检查下是否有倾斜，如果不是可以再用烙铁调整下。如果担心手抖，可以尝试使用尖头的烙铁，或者那种专业植入设备（虽然划不来）。</p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-45-45-IMG_2134.webp"></p><p>      植入完后是这样的，刚刚好能嵌入，没有材料被挤出。</p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-45-51-IMG_2136-1.webp"></p><p>      把所有的预留孔位都植入，前后面板一共28个</p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-45-59-IMG_2146-1.webp"></p><p>      这里是机箱的上盖，也需要植入两个螺母，用来固定风扇调速器。不过这里热熔螺母的尺寸会不同，为M2*3-3.2。</p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-46-05-IMG_2098-1.webp"></p><p>      主机底座，同样预留了m3螺丝孔，通过m3螺丝与主机固定。</p><h2 id="配置"><a href="#配置" class="headerlink" title="配置"></a>配置</h2><p>      这次准备了两套配置，虽然机箱兼容标准的thin itx主板，不过还是选择了两块有特色的主板，分别是过气网红DQ77KB搭配i5-3475s以及比较少见的华硕Q170T搭配QNVH魔改u。配置一股捡垃圾的气息，详细配置如下：</p><table><thead><tr><th></th><th>配置一</th><th>配置二</th></tr></thead><tbody><tr><td>主板:</td><td>Intel® Desktop Board DQ77KB</td><td>Asus Q170T</td></tr><tr><td>CPU:</td><td>i5-3475s,4c4t,6M Cache,up to 3.6GHz</td><td>QNVH(i7-8850h),6c12t,9M Cache,up to 3.6GHz</td></tr><tr><td>显卡:</td><td>Intel® HD Graphics 4000</td><td>Intel® UHD Graphics 630</td></tr><tr><td>散热器:</td><td>超微 SNK-P0046P LGA1150&#x2F;51&#x2F;55 + 建准 4010 12V&#x2F;0.8W 4500RPM x4</td><td>超微 SNK-P0046P LGA1150&#x2F;51&#x2F;55 + 建准 4020 12V&#x2F;0.8W 5200RPM x4</td></tr><tr><td>硬盘:</td><td>Intel 530 180G MSATA MLC SSD</td><td>海康 C2000pro 512gb</td></tr><tr><td>内存:</td><td>不知名 8G DDR3 1600 笔记本电脑内存条 x2</td><td>枭鲸 16G DDR4 2666 笔记本电脑内存条 x2</td></tr><tr><td>无线网卡:</td><td>-</td><td>Intel® Wi-Fi 6 AX200</td></tr><tr><td>电源:</td><td>HP 74&#x2F;50mm 19v 90w DC power adapter</td><td>Dell 74&#x2F;50mm 19v 130w DC power adapter</td></tr></tbody></table><p>      首先先介绍下过气网红Intel DQ77KB，一代神板，Thin ITX版型。这个2012年出厂的主板拥有双千兆网口、4个USB3.0、4个SATA接口、Mini PCIe、mSATA、 PCI-E x4接口，可谓是一应俱全，甚至放到现在都不算过时。而且其中一个网卡还支持AMT远程管理，远程修改BIOS，安装操作系统都没问题。所以不管是软路由，迷你服务器亦或者是跑个macOS都很适合。再加上当时200出头的售价，不要太吸引人。不过到了现在2021年，这块主板小黄鱼居然还能买的400多甚至600，妥妥的智商税。这块主板已经在角落默默服务了多年，陪我搬了几趟家换了几个城市，依旧一点问题都没有，不愧是Intel的原厂出品。感慨Intel都不再出主板了，如今把它掏出来，一是给他换个机箱改善散热，二是感觉他皮实，用它当小白鼠也不算心疼😏。       </p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-46-11-bad3f58b139fadee1d766ab4bc61f223.jpeg"></p><p>      接下来就是华硕的Q170T，可以理解为高配版的H110T，同样是双网卡、不同的是Q170T支持2280长度的NVME固态，而且速度是满速的PCI-e x4的速度，而不是阉割的x2速度。而且Q170芯片组支持Intel vPro技术，使用支持vPro的CPU可以实现类似于dq77kb的AMT远程管理。不过确定就是比较稀有，咸鱼刷了大半年有幸400蹲到一块。</p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-46-16-P_setting_xxx_0_90_end_692.webp"></p><h2 id="装机细节"><a href="#装机细节" class="headerlink" title="装机细节"></a>装机细节</h2><p>      这次选用了4010和4020两种风扇，下图展示的是4020风扇。原风扇的电线长度不够，自己更换了这种很便宜的超软红色硅胶线，26awg足量的话可以过3.5A的电流，妥妥的够用了。风扇通过螺丝固定在风扇架上。</p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-46-21-IMG_2196.webp"></p><p>      固定风扇用的是下图的这种夹板倒边螺母，用m3的规格，圆柱的外径是4mm。而4cm风扇的固定孔正好是4mm，这种螺母刚刚好可以放进风扇的固定孔，不会晃动。</p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-46-25-IMG_2328.webp"></p><p>      另一段用m3平头螺丝固定，风扇支架预留了位置，螺丝刚好可以沉进去不会突出。还可以看到风扇支架预留了理线槽，风扇的线可以统一从这里穿过去。</p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-46-40-IMG_2193.webp"></p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-46-43-IMG_2197.webp"></p><p>      接下来安装机箱的开关，因为不会画pcb板，所以用这种最笨的方式拔开关固定在3D打印的开关板上，再用导线接起来就好了，毕竟开机只是个短接的动作，没什么要求。</p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-46-53-IMG_2199.webp"></p><p>      开关板预留按钮元件的孔位，可以很稳的固定在上面，元件一边的针脚穿过开关板，用来固定元件。另一边的针脚用来焊接电线。</p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-46-56-IMG_2203.webp"></p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-47-00-IMG_2206.webp"></p><p>      然后就可以装在前面板了</p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-47-04-IMG_2208.webp"></p><p>      接下来固定主板，用4个8mm的m3铜柱。</p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-47-08-IMG_2215.webp"></p><p>      主板上提前安装好散热器，跟原版一样用的超微SNK-P0046P散热器。咸鱼拆机20一个不包邮，最低的时候我见过15不包。不过这个散热高度有点不足，会把主板拉变形。如果用魔改u会更明显。</p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-47-11-IMG_2217.webp"></p><p>      主板支架预留的沉头螺丝孔也刚好。</p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-47-16-IMG_2218.webp"></p><p>      接下来是安装主板的挡板，开孔试验过很多次，灰常很精准👌。如果觉得丑可以自己打印一个挡板装上去，一体感更强。</p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-50-51-IMG_2219.webp"></p><p>      拼装前先把小屏幕安装在机箱前面板上，同时链接好USB线。如果主板的USB接口位置比较刚刚，也需要先插好。</p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-51-20-IMG_2222%20%E6%8B%B7%E8%B4%9D.webp"></p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-51-53-IMG_2224.webp"></p><p>      然后就可以把三个部件连接起来</p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-53-12-IMG_2221.webp"></p><p>      再插上风扇。</p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-53-19-IMG_2225.webp"></p><p>      顶部面板装好风扇调速器，接好风扇再装上。</p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-53-27-IMG_2228.webp"></p><p>      装好顶部面板的效果，可以看到4020风扇还是有点荣誉的位置，可以再装个厚一点的4028。不过要论美观还是4010好看。</p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-53-36-IMG_2230.webp"></p><p>      后面板的两个洞是用来固定硬盘的，可以上一个9.5mm厚度的2.5寸硬盘。</p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-53-44-IMG_2231.webp"></p><p>      再装上两片10块的亚克力侧板就完成了</p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-53-51-IMG_2233.webp"></p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-53-56-IMG_2234.webp"></p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-54-25-IMG_2236%20%E6%8B%B7%E8%B4%9D.webp"></p><p>      最后再拧上支架。</p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-54-30-IMG_2237.webp"></p><p>      最后看看效果，很做作的买了4个芳生螺丝。家里猫多，而且这个机子其实我已经跑了一段时间，有点落灰，看起来脏脏的，各位看官见谅。</p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-54-51-IMG_2241%20%E6%8B%B7%E8%B4%9D.webp"></p><p>      Q170T主板装的黑色版本，来个黑白双煞。PLA材质颜色很多，机箱颜色可以任君组合。</p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/2021/12/17-15-55-05-IMG_2321%20%E6%8B%B7%E8%B4%9D.webp"></p><h2 id="补充说明"><a href="#补充说明" class="headerlink" title="补充说明"></a>补充说明</h2><ol><li><p>目前测试下来，4个0.8w的4010风扇是压不住这块QNCT处理器的，烤鸡会导致降频。一个是0.8w的4010的风扇风量不够，二是感觉这块风扇调速版输出电压会低不少（估计有1.5V）。好处是声音一点都没有，坏处就是即便调到最大功率，风扇也无法满速，风量完全不足。那4块4020的1.6w风扇的倒是能压住i5-3475s的烤鸡，明显风量大很多，热量呼呼的往外吹。感觉要用4010风扇的话需要选择更大功率的版本。</p></li><li><p>关于PLA材质耐热的问题，目前测试是没问题的。因为热源并不会直接接触到PLA材质，长时间的烤鸡也不至于达到PLA软化的温度。相比起耐热的问题，PLA的寿命问题更严重，几年之后PLA应该就脆得一捏就碎了。不过问题都很好解决，更换成ABS材料或者PETG材料就可以了。</p></li><li><p>Q170T用魔改U就是浪费了，因为会导致Vpro功能无法使用。不过垃圾佬买不起，关于Vpro的远程控制只能下次一定了。</p></li><li><p>图纸还是有很多提升的空间，所以暂时不考虑开源。主要是没想到更好的分享方式，虽然并不打算卖图纸，也担心被拿去打印卖钱，So……</p></li></ol>]]>
    </content>
    <id>https://blog.malu.tech/ThinItxCaseMK1/</id>
    <link href="https://blog.malu.tech/ThinItxCaseMK1/"/>
    <published>2021-12-15T16:00:00.000Z</published>
    <summary>许久前在CHH看到赵总设计的这款机箱后，就深深中毒了。奈何这片文章发表与2015年，且据闻这个机箱的产量极少，意味着想拥有一台几乎不可能了。...</summary>
    <title>复刻！赵总MK1，一个3D打印版的Thin-ITX机箱</title>
    <updated>2021-12-19T16:00:00.000Z</updated>
  </entry>
  <entry>
    <author>
      <name>Road</name>
    </author>
    <category term="mini主机" scheme="https://blog.malu.tech/categories/mini%E4%B8%BB%E6%9C%BA/"/>
    <category term="装机" scheme="https://blog.malu.tech/tags/%E8%A3%85%E6%9C%BA/"/>
    <category term="迷你主机" scheme="https://blog.malu.tech/tags/%E8%BF%B7%E4%BD%A0%E4%B8%BB%E6%9C%BA/"/>
    <category term="Opencore" scheme="https://blog.malu.tech/tags/Opencore/"/>
    <category term="Hackintosh" scheme="https://blog.malu.tech/tags/Hackintosh/"/>
    <category term="1L小主机" scheme="https://blog.malu.tech/tags/1L%E5%B0%8F%E4%B8%BB%E6%9C%BA/"/>
    <category term="4770hq" scheme="https://blog.malu.tech/tags/4770hq/"/>
    <category term="HP 800G1" scheme="https://blog.malu.tech/tags/HP-800G1/"/>
    <content>
      <![CDATA[<h2 id="BB两句"><a href="#BB两句" class="headerlink" title="BB两句"></a>BB两句</h2><p>HP 800G1 DM，在Haswell时代的1L小主机里是很有特色的存在，虽然比起同平台的联想（M73、M93）、戴尔（3020M，9020M）等的1L小主机丑了不少（PS.这个800G1看起来真的满满的年代感，很显旧），但是他很独特的选用了M.2接口而不是miniPCI。 而且相比起其他选用了M.2接口的小主机，他的M.2接口支持NVME协议（魔改后），而不是SATA协议。 虽然这个NVME协议速度并没有达到X4满速率，但是速度也比msata协议快不少。同时支持2280长度让硬盘的选择也很自由。 无线网卡的接口也是M.2且空间很充裕，可以轻松的通过转接卡用上白果的拆机网卡，不需要注入驱动就可以在macOS下实现隔空投送、接力、手表解锁等。 最后通过往BIOS注入微码可以使用如4980hq、4770hq等移动端魔改U，这类魔改u的4代i7，4c8t的配置，放在当时400左右的价格，性能并不会比同价位的6-8代的CPU（i3 8100之类）相差太多。 虽然4代CPU用的都是22nm工艺，会比14nm+++的工艺热不少，但是这类魔改u普遍搭载的是iris 5200核显，不仅黑苹果可以较完美的驱动，且性能要比祖传UHD630好不少（iris5200核显拥有40EU和128m的L4缓存，hd630仅24EU）。  综合以上数点，在当时（购入与2019年底）看来，不论是入坑macOS还是windows日常办公，这都是一个台很好玩且有性价比的小主机。</p><p>当然，放到现在已经是毫无性价比了。这台HP 800G1 DM我在19年底购入要360，4770hq魔改要420。而当时最便6代小主机要500+，最有性价比的ql2x好像也要480？（记不清楚了），所以在当时选择800G1还是有性价比的。到现在800G在tb还是这个价格，加上4代魔改i7全网没货，只剩下咸鱼传家宝开价500+。比起现在6代小主机比这个4代的800g1还便宜，而且还有神奇的6c12t的魔改8850H，只要450原。让这个800G1已经没有性价比了。没办法，谁叫我懒呢，折腾到现在才搞好macOS的安装。</p><p>总结一下，这台机子硬件方面的优势是：</p><ol><li>可以使用四代魔改移动的 CPU，比较低的价格就可以上 i7 八核（曾经）；</li><li>支持NVME协议、 2280长度的M.2硬盘；</li><li>同时还支持2.5寸Sata硬盘，总计可以安装两块硬盘；</li><li>网卡使用 ngff 接口且空间充裕，可以搭配转接卡使用用上白果的拆机网卡，完美驱动；</li><li>硬件保有量比较大，不用担心被JS涨价（曾经）；</li><li>噪音意外的小；</li></ol><h2 id="硬件"><a href="#硬件" class="headerlink" title="硬件"></a>硬件</h2><p>|                     | Specifications &#x2F; 型号              | Note &#x2F; 备注  |<br>| ———————|:————————————:|:————:|<br>| Motherboard&#x2F;主板:  | HP 800 G1                        |              |<br>| CPU&#x2F;处理器:           | I7-4770hq                           |            |<br>| CPU Cooler&#x2F;散热器:    | 自带                           |            |<br>| Hard Drive&#x2F;硬盘:     | Toshiba RD500 256gb                  |            |<br>| RAM&#x2F;内存:            | xiede 8G DDR3L 1600MHz X2         |            |<br>| Wireless Card&#x2F;无线网卡:| BCM94360CS2                   | 苹果拆机卡  |<br>| Tower Case&#x2F;机箱:      | 自带                                |            |<br>| Power&#x2F;电源:           | 7.4&#x2F;5.5mm 19v 90w DC power adapter |            |</p><h2 id="EFI下载地址"><a href="#EFI下载地址" class="headerlink" title="EFI下载地址"></a>EFI下载地址</h2><p>跳转至Github  <a href="https://github.com/Road-tech/Hackintosh_HP-800G1_I7-4770hq_OC">下载地址</a></p><p><strong>使用EFI前请务必修改三码(SSN,UUID,ROM)</strong><br><strong>Please change three system codes (SSN,UUID,ROM) before using this EFI</strong>   </p><h2 id="macOS完善情况"><a href="#macOS完善情况" class="headerlink" title="macOS完善情况"></a>macOS完善情况</h2><h3 id="支持："><a href="#支持：" class="headerlink" title="支持："></a>支持：</h3><ul><li>两个DP接口输出(1080p)  </li><li>所有的USB接口  </li><li>Wi-Fi &amp; Bluetooth  </li><li>3.5mm音频接口</li><li>机箱内置音响</li><li>Airdrop  </li><li>AirPlay  </li><li>Continuity  </li><li>睡眠</li><li>CPU变频</li></ul><h3 id="不支持"><a href="#不支持" class="headerlink" title="不支持:"></a>不支持:</h3><ul><li>VGA接口</li></ul><h3 id="未测试"><a href="#未测试" class="headerlink" title="未测试:"></a>未测试:</h3><ul><li>4k 输出</li><li>3.5mm麦克风输入</li></ul><h2 id="BIOS设定："><a href="#BIOS设定：" class="headerlink" title="BIOS设定："></a>BIOS设定：</h2><ul><li>Security -&gt; VTd -&gt; Disabled。 </li><li>Storage -&gt; Storage Options -&gt; SATA Emulation &gt; AHCI</li></ul><h2 id="禁用CFG-Lock-设定DVMT-pre-alloc到64M"><a href="#禁用CFG-Lock-设定DVMT-pre-alloc到64M" class="headerlink" title="禁用CFG Lock &amp; 设定DVMT pre-alloc到64M"></a>禁用CFG Lock &amp; 设定DVMT pre-alloc到64M</h2><p>需要一定动手能力，请参考<a href="https://www.bilibili.com/read/cv4646116/">不刷BIOS修改AMI BIOS的方法（以CFG Lock为例）</a></p><h2 id="Performance-展示"><a href="#Performance-展示" class="headerlink" title="Performance&#x2F;展示"></a>Performance&#x2F;展示</h2><p>待上传</p><h2 id="Reference-参考"><a href="#Reference-参考" class="headerlink" title="Reference&#x2F;参考"></a>Reference&#x2F;参考</h2><ul><li><a href="https://github.com/shidongfei2/800g1">https://github.com/shidongfei2/800g1</a></li><li><a href="https://github.com/zearp/OptiHack">https://github.com/zearp/OptiHack</a></li><li><a href="https://github.com/mingcheng/dell-optiplex-9020m-hackintosh">https://github.com/mingcheng/dell-optiplex-9020m-hackintosh</a></li><li><a href="https://www.bilibili.com/read/cv4646116/">https://www.bilibili.com/read/cv4646116/</a></li></ul>]]>
    </content>
    <id>https://blog.malu.tech/Hackintosh_HP-800G1_I7-4770hq_OC/</id>
    <link href="https://blog.malu.tech/Hackintosh_HP-800G1_I7-4770hq_OC/"/>
    <published>2021-04-19T16:00:00.000Z</published>
    <summary>买了很久的小主机，之前尝试使用安装苹果系统，OC引导一直不成功，只能使用Clover引导，最近在GitHub上发现有大佬分享了800G1的OC引导，就拿来尝试了下...</summary>
    <title>1L小主机系列 - HP 800G1</title>
    <updated>2021-04-19T16:00:00.000Z</updated>
  </entry>
  <entry>
    <author>
      <name>Road</name>
    </author>
    <category term="mini主机" scheme="https://blog.malu.tech/categories/mini%E4%B8%BB%E6%9C%BA/"/>
    <category term="装机" scheme="https://blog.malu.tech/tags/%E8%A3%85%E6%9C%BA/"/>
    <category term="迷你主机" scheme="https://blog.malu.tech/tags/%E8%BF%B7%E4%BD%A0%E4%B8%BB%E6%9C%BA/"/>
    <category term="Opencore" scheme="https://blog.malu.tech/tags/Opencore/"/>
    <category term="Hackintosh" scheme="https://blog.malu.tech/tags/Hackintosh/"/>
    <category term="Diy Mac mini" scheme="https://blog.malu.tech/tags/Diy-Mac-mini/"/>
    <category term="QNVH" scheme="https://blog.malu.tech/tags/QNVH/"/>
    <content>
      <![CDATA[<h2 id="BB两句"><a href="#BB两句" class="headerlink" title="BB两句"></a>BB两句</h2><p>之前Diy的一台Mac mini，用的是H110T+QN8J的组合，但是奈何技术有限，一直不能完美的支持16线程。后来把QN8J卖掉了，这台mini又收藏了起来，OC引导便一直没有更新了。突然又想起来这个坑，与心不甘，还是想再试试，奈何那么就过去了，QN8J还要850+。感觉很不值，于是买了这个QNVH。其实更多人应该听说过QNCT，就是8850H这个u的魔改。奈何Mr.Su家的QNCT卖完了，另外一家的8850H似乎挑内存，上不了高频，于是在海鲜市场买了这个QNVH，商家说类似于QNCT，看了看CPU频率一样，也不挑内存，380一颗还便宜，于是就选择了这个U。没想到刷上卖家给的BIOS，重新配置了下最新的OC0.6.8，就直接进系统了，超线程直接开启，完美支持8核16线程。有过之前配置的经验，这次EFI的制作很轻松！</p><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/Hackintosh-AsusH110T-QN8J-I7-8700Tes-DW1820A-OC/018.webp" alt="Diy Mac mini"> </p><h2 id="Hardware-硬件"><a href="#Hardware-硬件" class="headerlink" title="Hardware&#x2F;硬件"></a>Hardware&#x2F;硬件</h2><table><thead><tr><th></th><th align="center">Specifications &#x2F; 型号</th><th align="center">Note &#x2F; 备注</th></tr></thead><tbody><tr><td>Motherboard&#x2F;主板:</td><td align="center">Asus H110T</td><td align="center">Thin ITX</td></tr><tr><td>CPU&#x2F;处理器:</td><td align="center">QNVH</td><td align="center">I7 8850h ES</td></tr><tr><td>CPU Cooler&#x2F;散热器:</td><td align="center">Nimitz Diy Mac Mini 3D-printing cooling kit</td><td align="center"><a href="https://m.tb.cn/h.VSRHxNg?sm=8e19ac">Nimitz 超盒 3D打印散热套件</a></td></tr><tr><td>Hard Drive&#x2F;硬盘:</td><td align="center">Customization SSD using SM2263XT and 512g Intel TLC NAND Flash</td><td align="center"><a href="https://m.tb.cn/h.VSA0n4u?sm=086db2">NVME 固态套料 主控板2263XT</a></td></tr><tr><td>RAM&#x2F;内存:</td><td align="center">SEIWHALE 16G DDR4 2666MHz X2</td><td align="center"><a href="https://item.taobao.com/item.htm?id=612747898988">枭鲸 16G DDR4 2666 笔记本电脑内存条</a></td></tr><tr><td>Wireless Card&#x2F;无线网卡:</td><td align="center">BCM94350ZAE</td><td align="center">DW1820A</td></tr><tr><td>Tower Case&#x2F;机箱:</td><td align="center">Mac mini teardown case拆机机箱</td><td align="center">Mac mini 拆机机箱</td></tr><tr><td>Power&#x2F;电源:</td><td align="center">Dell 74&#x2F;50mm 19v 130w DC power adapter</td><td align="center"></td></tr></tbody></table><h2 id="EFI下载地址"><a href="#EFI下载地址" class="headerlink" title="EFI下载地址"></a>EFI下载地址</h2><p>跳转至Github  <a href="https://github.com/Road-tech/Hackintosh-AsusH110T-QNVH-I7-8850H-DW1820A-OC/">下载地址</a></p><p><strong>使用EFI前请务必修改三码(SSN,UUID,ROM)</strong><br><strong>Please change three system codes (SSN,UUID,ROM) before using this EFI</strong>   </p><h2 id="macOS完善情况"><a href="#macOS完善情况" class="headerlink" title="macOS完善情况"></a>macOS完善情况</h2><h3 id="支持："><a href="#支持：" class="headerlink" title="支持："></a>支持：</h3><ul><li>HDMI port (1080p) ｜ HDMI接口</li><li>DP port (1080p) ｜ DP接口</li><li>Audio output on HDMI ｜ HDMI接口音频输出</li><li>All USB ports ｜ 所有USB接口</li><li>Wi-Fi &amp; Bluetooth ｜ Wi-Fi &amp; 蓝牙</li><li>Dual Network Interface Card ｜ 双千兆</li><li>3.5mm Audio Output &amp; Mic Input ｜ 3.5mm音频输出</li><li>Airdrop ｜ 隔空投送</li><li>AirPlay ｜ 投屏</li><li>Continuity ｜ 接力   </li><li>hyper-threading ｜ 超线程</li><li>已加载原生电源管理</li><li>CPU变频</li><li>HEVC硬解码</li></ul><h3 id="未测试"><a href="#未测试" class="headerlink" title="未测试:"></a>未测试:</h3><ul><li>Sleep ｜ 睡眠</li><li>4k display ｜ 4K输出  （我没有4K屏）</li></ul><h2 id="备份MAC地址"><a href="#备份MAC地址" class="headerlink" title="备份MAC地址:"></a>备份MAC地址:</h2><p>华硕H110T这块主板有两张网卡：</p><ul><li>Realtek® RTL8111H  </li><li>Intel® I219V</li></ul><p>华硕把Intel网卡的MAC地址写在了BIOS里面，但是在为了支持8代CPU去魔改BIOS之后，会丢失Intel网卡的MAC地址。 具体表现为Intel网卡的MAC地址会变成<code>88:88:87:88</code>，所以要先备份MAC地址。</p><p>有两个方法去查MAC地址：</p><ul><li>主板的内存槽上贴着MAC地址，Intel网卡通常是左边那个</li><li>Windows下用 <code>ipconfig</code> 或者 macOS&#x2F;Linux下用 <code>ifconfig</code>去查MAC地址</li></ul><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/Hackintosh-AsusH110T-QN8J-I7-8700Tes-DW1820A-OC/100.webp" alt="image">  </p><h2 id="刷入魔改BIOS"><a href="#刷入魔改BIOS" class="headerlink" title="刷入魔改BIOS"></a>刷入魔改BIOS</h2><p>BIOS在上图左下角红框的位置，拔下来刷入。<br><strong>建议使用编程器刷入BIOS！</strong></p><p>当然你也可以尝试软刷bios，具体教程和BIOS文件都打包在<a href="https://github.com/Road-tech/Hackintosh-AsusH110T-QNVH-I7-8850H-DW1820A-OC/raw/main/H110T-ASUS-4212.zip">H110T-ASUS-4212.zip</a></p><h2 id="恢复MAC地址"><a href="#恢复MAC地址" class="headerlink" title="恢复MAC地址"></a>恢复MAC地址</h2><p>请准备以下工具：</p><ul><li>任意容量的U盘      </li><li>Rufus          <a href="https://rufus.ie/zh_CN.html">下载地址</a>     </li><li>EEUPDATE     <a href="https://raw.githubusercontent.com/Road-tech/Hackintosh-AsusH110T-QN8J-I7-8700Tes-DW1820A-OC/master/Eeupdate.rar">下载地址</a></li></ul><p>恢复MAC地址流程：</p><ol><li>打开Rufus，格式化U盘，制作DOS启动盘。</li><li>解压并复制EEUPDATA文件到U盘（假设EEUPDATA放在MAC文件夹内）。</li><li>U盘插入机子，进入BIOS设定U盘启动，进入DOS系统。</li><li>输入 <code>dir</code> 查看文件。</li><li>输入  <code>cd MAC</code> 进去MAC文件夹。</li><li>输入  <code>eeupdate /nic=1 /mac=XXXXXXXXX</code> 恢复MAC地址，<code>XXXXXXXXX</code>为你的记录的MAC地址。</li><li>提示成功后重启，进任意操作系统查看网卡MAC地址是否恢复成功。</li></ol><h2 id="BIOS设定："><a href="#BIOS设定：" class="headerlink" title="BIOS设定："></a>BIOS设定：</h2><h3 id="Disable-禁用："><a href="#Disable-禁用：" class="headerlink" title="Disable&#x2F;禁用："></a>Disable&#x2F;禁用：</h3><ul><li>Fast Boot  </li><li>CFG Lock   </li><li>VT-d  </li><li>CSM  </li><li>Intel SGX</li></ul><h3 id="Enable-启用："><a href="#Enable-启用：" class="headerlink" title="Enable&#x2F;启用："></a>Enable&#x2F;启用：</h3><ul><li>Intel Virtualization Technology   </li><li>Above 4G decoding  </li><li>Hyper Threading </li><li>Serial Port</li></ul><h2 id="安装macos"><a href="#安装macos" class="headerlink" title="安装macos"></a>安装macos</h2><p>请准备以下工具：</p><ul><li>系统镜像：请自备 macOS Big Sur 11.2.3 安装镜像   </li><li>OC编辑工具：OpenCore Configurator <a href="https://mackie100projects.altervista.org/">下载地址</a>    </li><li>镜像写入工具：Etcher （Windows，macOS，Linux皆可运行） <a href="https://www.balena.io/etcher/">下载地址</a>    </li><li>我提供的OC引导的EFI：<a href="https://github.com/Road-tech/Hackintosh-AsusH110T-QNVH-I7-8850H-DW1820A-OC/releases/download/v1.0/EFI.zip">下载地址</a>    </li><li>准备一个大于10g的u盘</li></ul><p>安装过程我就不重复了，大家可以参考下<a href="https://post.smzdm.com/p/adwrg48d/">我之前的文章</a>。</p><p>安装完成后请记得模拟NVRAM：<br>请在安装完系统后将增加 <strong>LogoutHook</strong> 文件用于放置在任意位置。并且在终端输入：<br> <code>sudo defaults write com.apple.loginwindow LogoutHook /path/to/LogoutHook.command</code></p><p>比如你放在<strong>下载</strong>文件夹内：<br><code>sudo defaults write com.apple.loginwindow LogoutHook /Users/xjn/Documents/LogoutHook/LogoutHook.command</code></p><p>重启后，你会在&#x2F;EFI&#x2F;下看到nvram.plist，代表已经成功模拟了。</p><p><strong>！运行后不要删除补丁包 ！</strong></p><h2 id="More-Detail-安装细节"><a href="#More-Detail-安装细节" class="headerlink" title="More Detail&#x2F;安装细节"></a>More Detail&#x2F;安装细节</h2><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/Hackintosh-AsusH110T-QN8J-I7-8700Tes-DW1820A-OC/001.webp" alt="image"><br>全家福<br><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/Hackintosh-AsusH110T-QN8J-I7-8700Tes-DW1820A-OC/002.webp" alt="image"><br>QN8J，35w，6核12线程，1.6GHz默频<br><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/Hackintosh-AsusH110T-QN8J-I7-8700Tes-DW1820A-OC/003.webp" alt="image"><br>屏蔽+短接<br><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/Hackintosh-AsusH110T-QN8J-I7-8700Tes-DW1820A-OC/004.webp" alt="image"><br>硬件合体，不知道枭鲸知不知道他家的内存贴纸贴反了<br><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/Hackintosh-AsusH110T-QN8J-I7-8700Tes-DW1820A-OC/005.webp" alt="image"><br>请出尼米兹散热器<br><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/Hackintosh-AsusH110T-QN8J-I7-8700Tes-DW1820A-OC/006.webp" alt="image"><br>将弹簧放在风道主体，然后把纯铜散热器压上去<br><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/Hackintosh-AsusH110T-QN8J-I7-8700Tes-DW1820A-OC/007.webp" alt="image"><br>把散热背板放在主板后面，然后把风道主体压上去，最后上螺丝拧紧。<br>这一步超级反人类！假象一下，弹簧并不能固定在主体上，散热器也不能，你要把它倒扣在主板上还要保证主体-弹簧-散热之间不能移位。最后你还要确保主体的螺丝孔能够对得上散热器的背板。<br><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/Hackintosh-AsusH110T-QN8J-I7-8700Tes-DW1820A-OC/008.webp" alt="image"><br>这个安装反人类到我不想装第二次！ 所以请尽量保证机子可以正常开机后再进行装机。<br><strong>请务必注意四颗螺丝的受力尽量均匀且不会过紧，不然可能会压弯主板！</strong><br><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/Hackintosh-AsusH110T-QN8J-I7-8700Tes-DW1820A-OC/009.webp" alt="image"><br>装上后IO板之后就可以推进机箱了！同时别忘了接上Wi-Fi天线。<br><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/Hackintosh-AsusH110T-QN8J-I7-8700Tes-DW1820A-OC/010.webp" alt="image"><br>成功合体！推入过程不会太顺畅的，要按压一下，刚刚能推进去。<br><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/Hackintosh-AsusH110T-QN8J-I7-8700Tes-DW1820A-OC/011.webp" alt="image"><br>最后装上风扇，这里可以看到3d打印的纹路，很粗糙<br><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/Hackintosh-AsusH110T-QN8J-I7-8700Tes-DW1820A-OC/012.webp" alt="image"><br><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/Hackintosh-AsusH110T-QN8J-I7-8700Tes-DW1820A-OC/013.webp" alt="image"><br>接下来就是重量嘉宾，大佬设计打印的网孔底盖！<br><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/Hackintosh-AsusH110T-QN8J-I7-8700Tes-DW1820A-OC/014.webp" alt="image"><br>还是很精致漂亮的<br><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/Hackintosh-AsusH110T-QN8J-I7-8700Tes-DW1820A-OC/015.webp" alt="image"><br>完美装上，严丝合缝<br><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/Hackintosh-AsusH110T-QN8J-I7-8700Tes-DW1820A-OC/016.webp" alt="image"><br>换个角度<br><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/Hackintosh-AsusH110T-QN8J-I7-8700Tes-DW1820A-OC/017.webp" alt="image"><br>装上小辣椒天线<br><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/Hackintosh-AsusH110T-QN8J-I7-8700Tes-DW1820A-OC/018.webp" alt="image"><br><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/Hackintosh-AsusH110T-QN8J-I7-8700Tes-DW1820A-OC/019.webp" alt="image"> </p><h2 id="Performance-系统展示"><a href="#Performance-系统展示" class="headerlink" title="Performance&#x2F;系统展示"></a>Performance&#x2F;系统展示</h2><p><img src="https://npm.elemecdn.com/road-blog-figure-webp@1.0.1/Hackintosh-AsusH110T-QN8J-I7-8700Tes-DW1820A-OC/102.webp" alt="image"> </p><h2 id="Reference-参考"><a href="#Reference-参考" class="headerlink" title="Reference&#x2F;参考"></a>Reference&#x2F;参考</h2><p><a href="https://blog.daliansky.net/OpenCore-BootLoader.html">精解OpenCore</a> - <a href="https://blog.daliansky.net/">黑果小兵的部落阁 </a></p><p><a href="https://blog.xjn819.com/?p=543">使用OpenCore引导黑苹果</a> - <a href="https://blog.xjn819.com/">XJN</a> </p><p><a href="https://blog.xjn819.com/?p=7">Asrock deskmini 310-com hackintosh 10.14-10.15 EFI</a> - <a href="https://blog.xjn819.com/">XJN</a></p><p><a href="https://blog.daliansky.net/DW1820A_BCM94350ZAE-driver-inserts-the-correct-posture.html">DW1820A&#x2F;BCM94350ZAE&#x2F;BCM94356ZEPA50DX插入的正确姿势</a> - <a href="https://blog.daliansky.net/">黑果小兵的部落阁</a></p><p><a href="http://www.smxdiy.com/thread-1862-1-1.html">华硕 ASUS H110T 支持 8 代 9 代 Xeon BIOS</a> - <a href="http://www.smxdiy.com/space-uid-1196.html">D大</a></p>]]>
    </content>
    <id>https://blog.malu.tech/Hackintosh-AsusH110T-QNVH-I7-8850H-DW1820A-OC/</id>
    <link href="https://blog.malu.tech/Hackintosh-AsusH110T-QNVH-I7-8850H-DW1820A-OC/"/>
    <published>2021-04-18T16:00:00.000Z</published>
    <summary>之前Diy的一台Mac mini，用的是H110T+QN8J的组合，后来把QN8J卖掉了，便再也没更新过。没想到过了那么久，QN8J还是那么贵，于是乎选择了更有性价比的选择：QNVH</summary>
    <title>DIY Mac mini后续 - CPU更换至QNVH</title>
    <updated>2021-04-20T16:00:00.000Z</updated>
  </entry>
</feed>
