从

从百草园到三味书屋

Recent content on 从百草园到三味书屋

https://youngforever.tech/
https://youngforever.tech/index.xml (RSS订阅地址)

医学人工智能周刊7｜医疗人工智能算法的公平性

<h1 id="医疗人工智能算法的公平性" class="relative group">医疗人工智能算法的公平性 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e5%8c%bb%e7%96%97%e4%ba%ba%e5%b7%a5%e6%99%ba%e8%83%bd%e7%ae%97%e6%b3%95%e7%9a%84%e5%85%ac%e5%b9%b3%e6%80%a7" aria-label="Anchor">#</a></span></h1><p>医疗领域，发展和部署不公平的人工智能系统会破坏公平治疗<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup>。在各亚人群中对AI模型的评估揭露了患者诊断、治疗以及收费上的不公平性。本文中作者从医疗角度概述了机器学习公平性，并且讨论算法偏差（特别是在数据获取、变异以及内在标签变异性）如何在临床工作流中出现并导致医疗差异。同样回顾了通过去纠缠、联邦学习和模型可解释性来减轻偏差的新兴的技术，以及这些方法在发展基于AI软件作为医疗设备的作用。</p> <h2 id="解决医疗差异和不平等的意义" class="relative group">解决医疗差异和不平等的意义 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e8%a7%a3%e5%86%b3%e5%8c%bb%e7%96%97%e5%b7%ae%e5%bc%82%e5%92%8c%e4%b8%8d%e5%b9%b3%e7%ad%89%e7%9a%84%e6%84%8f%e4%b9%89" aria-label="Anchor">#</a></span></h2><p>医疗差异不平等的原因，有观察到和隐藏的危险因素，了解这些差异的根源将指导改善服务患者不足的医疗保健的决策政策。 基于含有混淆历史偏差的数据发展的算法可能造成差异伤害，评估和减轻算法造成的伤害是研究医疗机器学习的重要动力。</p> <h3 id="医疗差异" class="relative group">医疗差异 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e5%8c%bb%e7%96%97%e5%b7%ae%e5%bc%82" aria-label="Anchor">#</a></span></h3><ul> <li>遗传变异和群体特异性表型：人种</li> <li>社会因素：地位财富等</li> <li>医学概念变化：ICD编码</li> <li>数据获取变化：数据获取的设备变化</li> <li>未知的医疗疾病：covid</li> <li>不同发展水平的国家部署AI-based software as medical device</li> </ul> <h2 id="医疗公平衡量指标定义" class="relative group">医疗公平衡量指标定义 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e5%8c%bb%e7%96%97%e5%85%ac%e5%b9%b3%e8%a1%a1%e9%87%8f%e6%8c%87%e6%a0%87%e5%ae%9a%e4%b9%89" aria-label="Anchor">#</a></span></h2><h3 id="demographic-parity" class="relative group">Demographic parity <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#demographic-parity" aria-label="Anchor">#</a></span></h3><p>以关注的因素如种族划分亚组，在各亚组中模型阳性预测比例相等。</p> <h3 id="predictive-parity" class="relative group">Predictive parity <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#predictive-parity" aria-label="Anchor">#</a></span></h3><p>阳性预测值（PPVs）和阴性预测值在各亚组应该相等。</p> <h3 id="equalized-odds" class="relative group">Equalized odds <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#equalized-odds" aria-label="Anchor">#</a></span></h3><p>TPRs和FPRs在各亚组相等。</p> <h2 id="消除医疗ai不公开的技术" class="relative group">消除医疗AI不公开的技术 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e6%b6%88%e9%99%a4%e5%8c%bb%e7%96%97ai%e4%b8%8d%e5%85%ac%e5%bc%80%e7%9a%84%e6%8a%80%e6%9c%af" aria-label="Anchor">#</a></span></h2><p>通过<strong>预处理</strong>步骤调整现有算法，以盲化、增强或重新加权输入空间；</p> <ul> <li>importance weighting</li> </ul> <p>为了消除混淆因素的影响，<strong>处理中</strong>技术在模型中构建了一个非区分项，来惩罚学习保护特性中的区分特征；</p> <ul> <li>non-discrimination term</li> </ul> <p><strong>后处理</strong>通过改变模型的输出来满足公平性指标；</p> <ul> <li>pick thresholds for each group</li> <li>a calibration curve can be fitted for each group</li> </ul> <div class="footnotes" role="doc-endnotes"> <hr> <ol> <li id="fn:1"> <p>Chen R J, Wang J J, Williamson D F K, et al. Algorithmic fairness in artificial intelligence for medicine and healthcare[J]. Nature Biomedical Engineering, 2023, 7(6): 719-742.&#160;<a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> </ol> </div>

2024/10/28
阅读更多

如何阅读论文

<h2 id="introduction" class="relative group">Introduction <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#introduction" aria-label="Anchor">#</a></span></h2><p>Efficiently read a paper is a critical but rarely taught skill.</p> <p>Describes the three-pass approach and its use in doing a literature survey.</p> <h2 id="the-three-pass-approach" class="relative group">The three-pass approach <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#the-three-pass-approach" aria-label="Anchor">#</a></span></h2><p>The first pass gives you a general idea about the paper. The second pass lets you grasp the paper’s content, but not its details. The third pass helps you understand the paper in depth.</p> <h3 id="the-first-pass" class="relative group">The first pass <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#the-first-pass" aria-label="Anchor">#</a></span></h3><p>A quick scan to get a bird’s-eye view of the paper, take about <strong>five to ten minutes:</strong></p> <ul> <li>carefully read the title, abstract, and introduction</li> <li>read the section and sub-section headings, but ignore everything else</li> <li>read the conclusions</li> <li>glance over the references, mentally ticking off the ones you’ve already read</li> </ul> <p>At the end of the first pass, you should be able to answer the 5Cs:</p> <ul> <li>Category: what type of paper is this?</li> <li>Context: which other papers is it related to? which theoretical bases were used?</li> <li>Correctness: Do the assumptions appear to be valid?</li> <li>Contributions: what are the paper’s main contributions?</li> <li>Clarity: Is the paper well written</li> </ul> <p><strong>think as a reviewer:</strong> if a reviewer cannot understand the gist after one pass, the paper may be rejected.</p> <h3 id="the-second-pass" class="relative group">The second pass <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#the-second-pass" aria-label="Anchor">#</a></span></h3><p>read the paper with greater care, but ignore details such as proofs, take up to an hour.</p> <ul> <li>look carefully at the figures, diagrams and other illustrations</li> <li>mark relevant unread reference for further reading</li> </ul> <h3 id="the-third-pass" class="relative group">The third pass <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#the-third-pass" aria-label="Anchor">#</a></span></h3><p>Take about four or five hours for beginners, and about an hour for an experienced reader.</p> <ul> <li>virtually re-implement the paper</li> </ul> <h2 id="doing-a-literature-survey" class="relative group">Doing a literature survey <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#doing-a-literature-survey" aria-label="Anchor">#</a></span></h2><ul> <li>Use academic search engine to find <strong>there to five recent papers</strong> in the area <ul> <li>read their related work</li> <li><strong>a recent survey paper</strong></li> </ul> </li> <li>Find shared citations and repeated author names in the bibliography <ul> <li><strong>download the key papers (1)</strong></li> <li>find where the best researchers published recently</li> </ul> </li> <li>Go to the website for these top conferences or journals and look through their recent proceedings <ul> <li><strong>identify recent high-quality related work(2)</strong></li> <li>constitute the first version of survey from (1) and (2)</li> <li>obtain key paper you didn’t find in these papers, obtain and read it</li> </ul> </li> </ul> <h2 id="reference" class="relative group">Reference <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#reference" aria-label="Anchor">#</a></span></h2><p>Keshav S. How to Read a Paper.</p>

2024/4/9
阅读更多

Skip-gram模型（2）

<script> document.addEventListener("DOMContentLoaded", function() { renderMathInElement(document.body, { delimiters: [ {left: '$$', right: '$$', display: true}, {left: '$', right: '$', display: false}, {left: '\\(', right: '\\)', display: false}, {left: '\\[', right: '\\]', display: true} ], throwOnError : false }); }); </script> <blockquote> <p><a href="https://youngforever.tech/posts/20230205-skip-gram-part1/" target="_blank" rel="noreferrer">之前文章</a>介绍了skip-gram的原理，这篇文章给出模型的实现细节以及pytorch和tensorflow的实现。</p> </blockquote> <h2 id="回顾" class="relative group">回顾 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e5%9b%9e%e9%a1%be" aria-label="Anchor">#</a></span></h2><p>假如用余弦相似度来计算两个词的one-hot编码得到0，即不能编码词之间的相似性，所以有了word2vec的方法，包括skip-gram和CBOW。</p> <p>接前文，假如我们拥有10000个单词的词汇表，想嵌入300维的词向量，那么我们的<strong>输入-隐层权重矩阵</strong>和<strong>隐层-输出层的权重矩阵</strong>都会有 10000 x 300 = 300万个权重，在如此庞大的神经网络中进行梯度下降是相当慢的。更糟糕的是，你需要大量的训练数据来调整这些权重并且避免过拟合。百万数量级的权重矩阵和亿万数量级的训练样本意味着训练这个模型将会是个灾难。<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup> 所以在具体实践上有一些计算技巧。</p> <h2 id="计算" class="relative group">计算 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e8%ae%a1%e7%ae%97" aria-label="Anchor">#</a></span></h2><p>skip-gram模型基于单个输入来预测上下文，对于T个训练单词$w_1,&hellip;w_T$，即最大化以下概率$$\frac{1}{T}\sum_{t=1}^T\sum_{-c&lt;=j&lt;=c}logP(w_{t+j}|w_t)\tag1$$其中c为训练上下文的窗口大小，$p(w_{t+j}|w_t)$用softmax来计算$$p(w_o|w_i)=\frac{exp(v_{wo}^Tv_{wi})}{\sum_{w=1}^Wexp(v_{wo}^Tv_{wi})}$$其中$v_{wi}$为输入词向量的表征，在单词量W巨大的情况下（通常都有$10^5-10^7$），式1的计算开销巨大。</p> <p>在skip-gram实际算法中使用多种策略来减少模型的资源使用（内存）以及提高词向量表征质量<sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup></p> <ul> <li>负采样 <ul> <li>从隐藏层到输出的Softmax层的计算量很大，因为要计算所有词的Softmax概率，再去找概率最大的值。例如当我们用训练样本 ( input word: &ldquo;fox&rdquo;，output word: &ldquo;quick&rdquo;) 来训练我们的神经网络时，“ fox”和“quick”都是经过one-hot编码的。如果我们的vocabulary大小为10000时，在输出层，我们期望对应“quick”单词的那个神经元结点输出1，其余9999个都应该输出0。在这里，这9999个我们期望输出为0的神经元结点所对应的单词我们称为“negative” word。<strong>当使用负采样时，我们将随机选择一小部分的negative words（比如选5个negative words）来更新对应的权重</strong>, 我们也会对我们的“positive” word进行权重更新<sup id="fnref:3"><a href="#fn:3" class="footnote-ref" role="doc-noteref">3</a></sup>。在实践中，​通常使用的是unigram分布的平方根，​即词汇表中每个词的概率的0.75次方除以归一化常数来挑选负样本。</li> </ul> </li> <li>高频词进行抽样 <ul> <li>原因：高频词相对于低频词来说提供的信息少；高频词随着样本增多本身表示也不会发生太大变化</li> <li>使用概率P来丢掉一定的单词$$P(w)=1- \sqrt{\frac{t}{f(w_i)}}$$其中t为设定的阈值，$f(w_i)$为单词出现的频率，可以看到频率越高丢弃的概率越大，反之越小</li> </ul> </li> <li>单词组合成词组作为单个词处理 <ul> <li>原因：组合词有特定的意思，不是简单把单个词的表示聚合起来</li> <li>如何从文本中提取出词组研究不少，skip-gram文章选用了$$socre(w_i, w_j)=\frac{count(w_iw_j)-\delta}{count(w_i)*count(w_j)}$$其中$w_i, w_j$代表不同的单词，利用score得分与设定的阈值比较来确定是否为常见词组</li> </ul> </li> </ul> <h2 id="skip-gram-pytorch实现" class="relative group">skip-gram PyTorch实现 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#skip-gram-pytorch%e5%ae%9e%e7%8e%b0" aria-label="Anchor">#</a></span></h2><p>Word2vec skip-gram pytorch<sup id="fnref:4"><a href="#fn:4" class="footnote-ref" role="doc-noteref">4</a></sup></p> <p><a href="https://github.com/yongfanbeta/skip-gram/blob/main/skipgram-pytorch.ipynb" target="_blank" rel="noreferrer">skipgram-pytorch.ipynb</a></p> <h2 id="skip-gram-tensorflow实现" class="relative group">skip-gram Tensorflow实现 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#skip-gram-tensorflow%e5%ae%9e%e7%8e%b0" aria-label="Anchor">#</a></span></h2><p>Word2vec skip-gram tensorflow<sup id="fnref:5"><a href="#fn:5" class="footnote-ref" role="doc-noteref">5</a></sup></p> <p><a href="https://github.com/yongfanbeta/skip-gram/blob/main/skipgram-tf.ipynb" target="_blank" rel="noreferrer">skipgram-tf.ipynb</a></p> <div class="footnotes" role="doc-endnotes"> <hr> <ol> <li id="fn:1"> <p><a href="https://zhuanlan.zhihu.com/p/27234078" target="_blank" rel="noreferrer">理解 Word2Vec 之 Skip-Gram 模型 - 知乎</a>&#160;<a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:2"> <p><a href="https://arxiv.org/pdf/1310.4546.pdf" target="_blank" rel="noreferrer">Distributed Representations of Words and Phrases and their Compositionality</a>&#160;<a href="#fnref:2" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:3"> <p><a href="https://www.jianshu.com/p/cf36d4c1ea39" target="_blank" rel="noreferrer">关于skip-gram和负采样 - 简书</a>&#160;<a href="#fnref:3" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:4"> <p><a href="http://zh.d2l.ai/chapter_natural-language-processing-pretraining/word2vec.html" target="_blank" rel="noreferrer">14.1. 词嵌入（word2vec） — 动手学深度学习 2.0.0 documentation</a>&#160;<a href="#fnref:4" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:5"> <p><a href="https://www.tensorflow.org/tutorials/text/word2vec#negative_sampling_for_one_skip-gram" target="_blank" rel="noreferrer">word2vec  |  TensorFlow Core</a>&#160;<a href="#fnref:5" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> </ol> </div>

2023/7/12
阅读更多

医学人工智能周刊6｜模态无关的学习方法在医学影像以及生理信号中的评测

<h2 id="摘要" class="relative group">摘要 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e6%91%98%e8%a6%81" aria-label="Anchor">#</a></span></h2><p>目的：建立基准测试BenchMD，用于测试模型无关的方法包括<strong>架构</strong>和<strong>训练技术</strong>（例如自监督学习、预训练）在临床相关的医疗任务上的表现。简言之，就是测试最新一些通用人工智能方法在医疗任务上的表现。</p> <p>BenchMD包括19个公开数据集，7种医疗数据模态，1维传感器数据、2维图片、3维扫描数据。</p> <p>结果表明，没有一种与模态无关的技术在所有模态上都能实现强大的性能，基准模型有充足的改进空间。</p> <h2 id="引言" class="relative group">引言 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e5%bc%95%e8%a8%80" aria-label="Anchor">#</a></span></h2><p>背景：Transformers模型和自监督学习（SSL）对标签数据需求小、能灵活运用到多种模态的数据。</p> <p>问题：衡量这些进展在领域内的效果需要制定具有广度和深度的评测，以捕捉应用和模式的多样性，并通过让专家参与评测过程来确保外部有效性。</p> <p>当前医疗AI领域应用时针对具体问题，通过试验选择不同的架构以及自监督学习方法，期望发展一种灵活、与模态无关无需定制化就能应用到各类问题的方法。</p> <p>解决：BenchMD针对每种模态构建标准化、临床有效的评估方法，并通过专家验证；同时探索了基准<strong>数据标签不足</strong>情况和<strong>数据偏移</strong>情况下的表现；</p> <p>同时，为了让BenchMD更加容易使用：</p> <ul> <li>易用性：新架构和任务即插即用</li> <li>易复现：全部使用公开数据集</li> </ul> <p>结果显示医疗AI领域通用、泛化性强方法仍需继续研究。</p> <h2 id="相关工作" class="relative group">相关工作 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e7%9b%b8%e5%85%b3%e5%b7%a5%e4%bd%9c" aria-label="Anchor">#</a></span></h2><ul> <li>模态无关的技术：SSL <ul> <li>掩码建模</li> <li>对比学习</li> </ul> </li> <li>模态无关的医学人工智能 <ul> <li>在医学图像上自监督MAE比ImageNet上预训练要好</li> <li>医学影像有无监督预训练加上监督学习表现好</li> </ul> </li> <li>现有多种模态的基准测试 <ul> <li><a href="https://github.com/alextamkin/dabs" target="_blank" rel="noreferrer">GitHub - alextamkin/dabs: A Domain-Agnostic Benchmark for Self-Supervised Learning</a></li> <li><a href="https://github.com/p-lambda/wilds" target="_blank" rel="noreferrer">GitHub - p-lambda/wilds: A machine learning benchmark of in-the-wild distribution shifts, with data loaders, evaluators, and default models.</a></li> </ul> </li> </ul> <h2 id="模态和数据集" class="relative group">模态和数据集 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e6%a8%a1%e6%80%81%e5%92%8c%e6%95%b0%e6%8d%ae%e9%9b%86" aria-label="Anchor">#</a></span></h2><p>整理了一系列高影响模态数据以及精心挑选的数据源和目标数据集，用于评估分布外 （OOD） 性能。</p> <h3 id="12-lead-ecgs" class="relative group">12-lead ECGs <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#12-lead-ecgs" aria-label="Anchor">#</a></span></h3><p>利用5秒采样频率为500Hz的12导心电数据进行7分类：正常、传导障碍、心肌肥厚、心肌梗死、缺血性ST-T改变、心房颤动/心房扑动及其他。</p> <p>数据包括：</p> <ul> <li>PTB-XL (18k) 1989-1996 <ul> <li><a href="https://physionet.org/content/ptb-xl/1.0.3/" target="_blank" rel="noreferrer">PTB-XL, a large publicly available electrocardiography dataset v1.0.3</a></li> </ul> </li> <li>Chapman-Shaoxing (10k) 2020 <ul> <li><a href="https://figshare.com/collections/ChapmanECG/4560497/2" target="_blank" rel="noreferrer">A 12-lead electrocardiogram database for arrhythmia research covering more than 10,000 patients</a></li> </ul> </li> <li>Georgia 12-Lead ECG Challenge (10k) 2020 <ul> <li><a href="https://physionet.org/content/challenge-2020/1.0.2/#files" target="_blank" rel="noreferrer">Classification of 12-lead ECGs: The PhysioNet/Computing in Cardiology Challenge 2020 v1.0.2</a></li> </ul> </li> <li>China Physiological Signal Challenge (6.8k) 2018</li> </ul> <h3 id="eeg" class="relative group">EEG <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#eeg" aria-label="Anchor">#</a></span></h3><p>30秒单导EEG睡眠分期任务。使用AASM睡眠分期标准：觉醒、快速眼动、非快速眼动I期、非快速眼动2期、非快速眼动3期。</p> <p>数据包括：</p> <ul> <li>SHHS (5.8k) 1995-1998 <ul> <li>includes 5,804 adults aged 40 and older</li> <li><a href="https://biolincc.nhlbi.nih.gov/studies/shhs/" target="_blank" rel="noreferrer">BioLINCC: Sleep Heart Health Study (SHHS)</a></li> </ul> </li> <li>ISRUC-Sleep（0.1k）2009-2013 <ul> <li>collected from subjects in hospital whose ages range from 20 years old to 85 years old, with an average age of 51</li> <li><a href="https://sleeptight.isr.uc.pt/" target="_blank" rel="noreferrer">ISRUC-SLEEP Dataset | A comprehensive public dataset for sleep researchers</a></li> </ul> </li> </ul> <h3 id="chest-x-rays" class="relative group">Chest X-Rays <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#chest-x-rays" aria-label="Anchor">#</a></span></h3><p>使用2D灰度胸片进行单标签分类任务，包括肺不张、心脏扩大、实变、水肿和胸腔积液。</p> <p>数据包括：</p> <ul> <li>MIMIC-CXR (227k) 2011-2016 <ul> <li>a large publicly available dataset of chest radiographs in DICOM format with free-text radiology reports.</li> <li><a href="https://physionet.org/content/mimic-cxr/2.0.0/" target="_blank" rel="noreferrer">MIMIC-CXR Database v2.0.0</a></li> </ul> </li> <li>CheXpert (65k) 2002-2017 <ul> <li>a large public dataset for chest radiograph interpretation, consisting of 224,316 chest radiographs of 65,240 patients.</li> <li>[CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison](<a href="https://stanfordmlgroup.github.io/competitions/chexpert/" target="_blank" rel="noreferrer">https://stanfordmlgroup.github.io/competitions/chexpert/</a></li> </ul> </li> <li>VinDr-CXR (18k) 2018-2020 <ul> <li>The published dataset consists of 18,000 postero-anterior (PA) view CXR scans that come with both the localization of critical findings and the classification of common thoracic diseases.</li> <li><a href="https://vindr.ai/datasets/cxr" target="_blank" rel="noreferrer">VinDr-CXR: An open dataset and benchmarks for disease classification and abnormality localization on chest radiographs | VinDr</a></li> </ul> </li> </ul> <h3 id="mammograms" class="relative group">Mammograms <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#mammograms" aria-label="Anchor">#</a></span></h3><p>乳腺X光检查包括患者左右乳房不同视图的2D灰度图像从1-5类BI-RADS分级。</p> <p>数据包括：</p> <ul> <li>VinDr-Mammo (5k) 2018-2020 <ul> <li>a large-scale benchmark dataset of full-field digital mammography, called VinDr-Mammo</li> <li><a href="https://vindr.ai/datasets/mammo" target="_blank" rel="noreferrer">VinDr-Mammo: A large-scale benchmark dataset for computer-aided diagnosis in full-field digital mammography | VinDr</a></li> </ul> </li> <li>CBIS-DDSM (2.6k)1988-1999 <ul> <li>The DDSM is a database of 2,620 scanned film mammography studies. It contains normal, benign, and malignant cases with verified pathology information.</li> <li><a href="https://wiki.cancerimagingarchive.net/pages/viewpage.action?pageId=22516629" target="_blank" rel="noreferrer">Curated Breast Imaging Subset of Digital Database for Screening Mammography (CBIS-DDSM) - The Cancer Imaging Archive (TCIA) Public Access - Cancer Imaging Archive Wiki</a></li> </ul> </li> </ul> <h3 id="dermoscopic" class="relative group">Dermoscopic <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#dermoscopic" aria-label="Anchor">#</a></span></h3><p>基于2D RGB皮肤图像进行单标签分类，共5类：AKIEC“（包括光化性角化病、上皮内癌和鳞状细胞癌，因为所有这些都是鳞状细胞癌的连续体）、”BCC“（基底细胞癌）、”MEL“（黑色素瘤）、”NEV“（痣）和”其他疾病“（皮肤纤维瘤等）。</p> <p>数据包括：</p> <ul> <li>BCN20000（19k）2010-2016 <ul> <li>paper: <a href="https://arxiv.org/pdf/1908.02288.pdf" target="_blank" rel="noreferrer">BCN20000: DERMOSCOPIC LESIONS IN THE WILD</a></li> <li>data: <a href="https://challenge.isic-archive.com/data/#2019" target="_blank" rel="noreferrer">ISIC Challenge</a></li> </ul> </li> <li>HAM10000（10k） <ul> <li>paper: <a href="https://www.nature.com/articles/sdata2018161" target="_blank" rel="noreferrer">The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions | Scientific Data</a></li> <li>data: <a href="https://isic-archive.com/" target="_blank" rel="noreferrer">ISIC | International Skin Imaging Collaboration</a></li> </ul> </li> <li>PAD-UFES-20（1.37k）2018-2019 <ul> <li>a nonprofit program that provides free skin lesion treatment, in particular, to low-income people who cannot afford private treatment.</li> <li>paper: <a href="https://arxiv.org/abs/2007.00478" target="_blank" rel="noreferrer">[2007.00478] PAD-UFES-20: a skin lesion dataset composed of patient data and clinical images collected from smartphones</a></li> <li>data: <a href="https://data.mendeley.com/datasets/zr7vgbcyr2/1" target="_blank" rel="noreferrer">PAD-UFES-20: a skin lesion dataset composed of patient data and clinical images collected from smartphones - Mendeley Data</a></li> </ul> </li> </ul> <h3 id="fundus-images" class="relative group">Fundus Images <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#fundus-images" aria-label="Anchor">#</a></span></h3><p>基于2D RGB眼底图像预测糖尿病视网膜病变严重程度，基于ICDR分级共5类</p> <p>数据包括：</p> <ul> <li>Messidor-2（0.5k）2004-2010 <ul> <li>a collection of Diabetic Retinopathy (DR) examinations, each consisting of two macula-centered eye fundus images (one per eye)</li> <li><a href="https://www.adcis.net/en/third-party/messidor2/" target="_blank" rel="noreferrer">Messidor-2 - ADCIS</a></li> </ul> </li> <li>APTOS 2019（3.6k）2019 <ul> <li>3662 samples collected from many participants of rural India</li> <li><a href="https://www.kaggle.com/competitions/aptos2019-blindness-detection/data" target="_blank" rel="noreferrer">APTOS 2019 Blindness Detection | Kaggle</a></li> </ul> </li> <li>Jinchi Medical University dataset（2.7k）2011-2015 <ul> <li>paper：<a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5480986/#:~:text=Here%2C%20we%20show%20an%20AI%20that%20grades%20diabetic,staging%20and%20can%20suggest%20treatments%20and%20predict%20prognoses." target="_blank" rel="noreferrer">Applying artificial intelligence to disease staging: Deep learning for improved staging of diabetic retinopathy - PMC</a></li> <li>data：<a href="https://figshare.com/articles/figure/Davis_Grading_of_One_and_Concatenated_Figures/4879853/1" target="_blank" rel="noreferrer">Davis Grading of One and Concatenated Figures</a></li> </ul> </li> </ul> <h3 id="low-dose-computer-tomography-scansldct" class="relative group">Low Dose Computer Tomography Scans（LDCT） <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#low-dose-computer-tomography-scansldct" aria-label="Anchor">#</a></span></h3><p>基于3D CT影像进行判断结节大小</p> <p>数据包括：</p> <ul> <li>LIDC-IDRI（1.0k）2010 <ul> <li><a href="https://wiki.cancerimagingarchive.net/pages/viewpage.action?pageId=1966254" target="_blank" rel="noreferrer">Data from The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): A completed reference database of lung nodules on CT scans (LIDC-IDRI) - The Cancer Imaging Archive (TCIA) Public Access - Cancer Imaging Archive Wiki</a></li> </ul> </li> <li>LNDb（294）2016-2018 <ul> <li><a href="https://zenodo.org/record/7153205" target="_blank" rel="noreferrer">LNDb Dataset | Zenodo</a></li> </ul> </li> </ul> <h2 id="实验" class="relative group">实验 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e5%ae%9e%e9%aa%8c" aria-label="Anchor">#</a></span></h2><p>对5种技术进行评估：3种SSL算法、ImageNet预训练、从头训练，然后使用多种迁移学习方法测试在分布外（OOD）数据中性能。</p> <h3 id="网络架构" class="relative group">网络架构 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e7%bd%91%e7%bb%9c%e6%9e%b6%e6%9e%84" aria-label="Anchor">#</a></span></h3><p>分别使用1D、2D和3D的embedding模块处理原始数据形成256维度的嵌入空间，不同输入维度的信息能混合。编码器使用的是标准的ViT架构。</p> <h3 id="预训练" class="relative group">预训练 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e9%a2%84%e8%ae%ad%e7%bb%83" aria-label="Anchor">#</a></span></h3><p>三种自监督方法</p> <ul> <li>Contrastive embedding-Mixup(e-Mix) <ul> <li>使用一定的比例系数对原始输入嵌入加权并相加，训练编码器为混合输入产生一个向量，该向量与原始输入经过混合因子加权相加尽量相近</li> </ul> </li> <li>Shuffled embedding prediction(ShED) <ul> <li>打乱一部分输入嵌入，使用带分类器的编码器来预测被扰动过的嵌入</li> </ul> </li> <li>MAE <ul> <li>对输入嵌入表达进行75%的掩码，训练模型重建输入对嵌入表达</li> </ul> </li> </ul> <h3 id="迁移学习以及分布外评价" class="relative group">迁移学习以及分布外评价 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e8%bf%81%e7%a7%bb%e5%ad%a6%e4%b9%a0%e4%bb%a5%e5%8f%8a%e5%88%86%e5%b8%83%e5%a4%96%e8%af%84%e4%bb%b7" aria-label="Anchor">#</a></span></h3><p>固定模型骨架利用分布内数据训练一个线性分类器进行微调。然后在分布外数据集中进行zero-shot评估。微调数据集的选取，单标签任务选取，多标签任务选取。</p> <p> <figure><img src="https://cdn.jsdelivr.net/gh/jmwyf/pichosting@master/benchmdpng" alt="" class="mx-auto my-0 rounded-md" /> </figure> </p> <h2 id="结果" class="relative group">结果 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e7%bb%93%e6%9e%9c" aria-label="Anchor">#</a></span></h2><ul> <li>各类自监督方法在各模态数据上表现不一致，需要探索在各模态数据中表现更加一致的SSL算法</li> <li>SSL有时优于预训练方法，有时预训练也可以和SSL表现相当。<strong>未来需要探索在其他数据集例如imagenet中进行预训练，然后在医疗数据集中进行SSL</strong>，即预训练与自监督结合。</li> <li>微调过程中标化数据量影响模型性能，越多越好，但也要防止过拟合的情况发生。</li> <li>分布内与分布外数据模型性能比较表明需要探索提升模型可泛化性能的正则化技术</li> </ul> <h2 id="思考" class="relative group">思考 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e6%80%9d%e8%80%83" aria-label="Anchor">#</a></span></h2><p>自监督技术和预训练技术在NLP和CV领域应用广泛，如何应用到各类医疗数据中，是不是在所有种类的医疗数据中表现都比不使用要好，本文通过构建各类基准模型尝试回答该问题，相比于NLP和CV领域，医疗领域数据种类繁多导致当前没有一种统一的方法适用于所有的情形，需要进一步研究判断对各种类的数据适合使用的方法，以及预训练与自监督技术联合使用的方法。</p> <blockquote> <p>Wantlin, K. et al. BenchMD: A Benchmark for Modality-Agnostic Learning on Medical Images and Sensors. Preprint at <a href="https://doi.org/10.48550/arXiv.2304.08486" target="_blank" rel="noreferrer">https://doi.org/10.48550/arXiv.2304.08486</a> (2023).</p> </blockquote>

2023/7/10
阅读更多

医学人工智能周刊5｜提高医疗领域AI算法研究透明性清单

<h2 id="有关临床人工智能建模的最小信息mi-claim检查清单" class="relative group">有关临床人工智能建模的最小信息：MI-CLAIM检查清单 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e6%9c%89%e5%85%b3%e4%b8%b4%e5%ba%8a%e4%ba%ba%e5%b7%a5%e6%99%ba%e8%83%bd%e5%bb%ba%e6%a8%a1%e7%9a%84%e6%9c%80%e5%b0%8f%e4%bf%a1%e6%81%afmi-claim%e6%a3%80%e6%9f%a5%e6%b8%85%e5%8d%95" aria-label="Anchor">#</a></span></h2><p>一种用于提高医疗领域AI算法透明性报告的工具</p> <h3 id="研究设计" class="relative group">研究设计 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e7%a0%94%e7%a9%b6%e8%ae%be%e8%ae%a1" aria-label="Anchor">#</a></span></h3><ul> <li><input disabled="" type="checkbox"> 模型解决的临床问题在论文中有清楚的细节</li> <li><input disabled="" type="checkbox"> 清晰的表述研究问题</li> <li><input disabled="" type="checkbox"> 患者队列的特征有细节文本体现</li> <li><input disabled="" type="checkbox"> 患者队列能代表真实世界临床环境</li> <li><input disabled="" type="checkbox"> 确定并详细说明用于基线比较最先进的解决方案</li> </ul> <h3 id="数据和优化" class="relative group">数据和优化 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e6%95%b0%e6%8d%ae%e5%92%8c%e4%bc%98%e5%8c%96" aria-label="Anchor">#</a></span></h3><ul> <li><input disabled="" type="checkbox"> 原始数据以及原始格式在文中描述清楚</li> <li><input disabled="" type="checkbox"> 用于模型的数据转化被详细描述</li> <li><input disabled="" type="checkbox"> 训练集和测试集说吗相互独立</li> <li><input disabled="" type="checkbox"> 描述关于如何选择最优模型的细节</li> <li><input disabled="" type="checkbox"> 输入数据是结构化还是非结构化</li> </ul> <h3 id="模型性能" class="relative group">模型性能 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e6%a8%a1%e5%9e%8b%e6%80%a7%e8%83%bd" aria-label="Anchor">#</a></span></h3><ul> <li><input disabled="" type="checkbox"> 描述用于评价算法性能（AUC、F-score等）的主要指标，包括选择依据</li> <li><input disabled="" type="checkbox"> 描述用于评价模型临床利用（PPV、NNT等）的主要指标，包括选择依据</li> <li><input disabled="" type="checkbox"> 使用合适的统计学方法描述比较提出模型性能与基线模型的性能</li> </ul> <h3 id="模型测试" class="relative group">模型测试 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e6%a8%a1%e5%9e%8b%e6%b5%8b%e8%af%95" aria-label="Anchor">#</a></span></h3><ul> <li><input disabled="" type="checkbox"> 检查技术1，结构化数据系数以及敏感度分析</li> <li><input disabled="" type="checkbox"> 检查技术2，非结构化数据显著图以及敏感度分析</li> <li><input disabled="" type="checkbox"> 讨论模型/算法性能与检查结果相关性</li> <li><input disabled="" type="checkbox"> 讨论在检查方法不可解释的情况下，在案例层面模型可解释性的可行性与意义</li> <li><input disabled="" type="checkbox"> 讨论在底层数据分布偏移情况下模型的可靠性以及鲁棒性</li> </ul> <h3 id="可重复性" class="relative group">可重复性 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e5%8f%af%e9%87%8d%e5%a4%8d%e6%80%a7" aria-label="Anchor">#</a></span></h3><ul> <li><input disabled="" type="checkbox"> 完成代码分享</li> <li><input disabled="" type="checkbox"> 允许第三方评测代码的准确性/公平性；分享评测结果</li> <li><input disabled="" type="checkbox"> 发布用于在新数据上运行代码而不共享其详细信息的虚拟机(二进制文件)</li> <li><input disabled="" type="checkbox"> 不分享</li> </ul> <p><a href="https://www.nature.com/articles/s41591-020-1041-y" target="_blank" rel="noreferrer">Minimum information about clinical artificial intelligence modeling: the MI-CLAIM checklist | Nature Medicine</a></p>

2023/6/23
阅读更多

医学人工智能周刊4｜如何解决医学人工智能的可解释性

<h2 id="构建临床医生需求与开发目标之间的桥梁来解决医学人工智能可解释性" class="relative group">构建临床医生需求与开发目标之间的桥梁来解决医学人工智能可解释性 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e6%9e%84%e5%bb%ba%e4%b8%b4%e5%ba%8a%e5%8c%bb%e7%94%9f%e9%9c%80%e6%b1%82%e4%b8%8e%e5%bc%80%e5%8f%91%e7%9b%ae%e6%a0%87%e4%b9%8b%e9%97%b4%e7%9a%84%e6%a1%a5%e6%a2%81%e6%9d%a5%e8%a7%a3%e5%86%b3%e5%8c%bb%e5%ad%a6%e4%ba%ba%e5%b7%a5%e6%99%ba%e8%83%bd%e5%8f%af%e8%a7%a3%e9%87%8a%e6%80%a7" aria-label="Anchor">#</a></span></h2><p>可解释人工智能（XAI）已成为AI/ML在医疗领域实践应用一种有前途的解决方法。然而，开发者和临床人员如何解释XAI，以及他们之间冲突的目标和需求不太清楚。这篇文章介绍了一项纵向多种方法（调查问卷、实验、采访）的研究结果，该研究涉及112个开发人员和临床人员共同设计用于临床决策支持系统（CDSS）的XAI解决方案。本研究确定了XAI在开发者和临床医生心智模型中3个重要的差别，包括对立的目标（模型可解释性与临床合理性），不同事实来源（数据vs患者），以及探索新知识和利用旧知识。基于上述发现，文中提出能帮助解决医疗领域XAI难题的解决方案，包括利用因果推断模型，个性化解释以及探索和利用间的平衡。本研究强调在XAI系统设计中同时考虑开发者和临床医生角度观点的重要性并且提供提高医疗领域XAI有效性和可用性的实用建议。</p> <p> <figure><img src="https://cdn.jsdelivr.net/gh/jmwyf/pichosting@master/XAI.png" alt="" class="mx-auto my-0 rounded-md" /> </figure> </p> <p><a href="https://www.nature.com/articles/s41746-023-00837-4" target="_blank" rel="noreferrer">Solving the explainable AI conundrum by bridging clinicians’ needs and developers’ goals | npj Digital Medicine</a></p> <h2 id="医学影像人工智能解释的现状与未来" class="relative group">医学影像人工智能解释的现状与未来 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e5%8c%bb%e5%ad%a6%e5%bd%b1%e5%83%8f%e4%ba%ba%e5%b7%a5%e6%99%ba%e8%83%bd%e8%a7%a3%e9%87%8a%e7%9a%84%e7%8e%b0%e7%8a%b6%e4%b8%8e%e6%9c%aa%e6%9d%a5" aria-label="Anchor">#</a></span></h2><p>医学影像的解释是放射科医生工作的核心任务，近年来人工智能应用逐渐参与其中。这篇文章回顾了影像人工智能模型发展及其在临床实践中应用的进展、挑战和机遇。文章讨论了AI算法帮助放射科医生完成包括检查、工作流分类以及量化的功能，也讨论了非影像科医生使用医学影像AI的趋势。文章指出了在放射学使用AI算法时泛化的核心挑战，以及对包含临床医生-AI合作、透明度以及部署后监测的验证保障措施的需求。最后，文章讨论了人工智能领域多模态LLM的快速发展，这一进展为开发通用医疗人工智能模型提供了重大机遇，这些模型可以处理各种医学影像解释任务等。</p> <p> <figure><img src="https://cdn.jsdelivr.net/gh/jmwyf/pichosting@master/AIinterpretation.png" alt="" class="mx-auto my-0 rounded-md" /> </figure> </p> <p><a href="https://doi.org/10.1056/NEJMra2301725" target="_blank" rel="noreferrer">The Current and Future State of AI Interpretation of Medical Images | NEJM</a></p> <h2 id="基于深度学习方法从疾病轨迹中预测胰腺癌风险" class="relative group">基于深度学习方法从疾病轨迹中预测胰腺癌风险 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e5%9f%ba%e4%ba%8e%e6%b7%b1%e5%ba%a6%e5%ad%a6%e4%b9%a0%e6%96%b9%e6%b3%95%e4%bb%8e%e7%96%be%e7%97%85%e8%bd%a8%e8%bf%b9%e4%b8%ad%e9%a2%84%e6%b5%8b%e8%83%b0%e8%85%ba%e7%99%8c%e9%a3%8e%e9%99%a9" aria-label="Anchor">#</a></span></h2><p>胰腺癌是一种侵袭性疾病，通常表现较晚结局较差，这意味着需要早期诊断。本研究中，将人工智能方法应用在丹麦（丹麦国家患者登记处，DNPR）六百万患者（24000胰腺癌患者病例）和美国（美国退伍军人事务部，US-VA）三百万患者（3900胰腺癌病例）的临床数据中，在临床病史的疾病代码序列中训练机器学习模型，并在逐渐增加的时间窗口中测试预测癌症发生率。对于在36个月内的发生癌症，DNPR数据最佳模型性能为AUROC=0.88，当在癌症诊断前3个月内的疾病事件从训练机中排除时，AUROC（3m）减少至0.83，1000个年龄高于50岁的高风险人群估计相对风险为59。将丹麦模型交叉验证到US-VA中有较低的性能表现（AUROC=0.71），可以通过再训练来提高模型性能。这些结果说明为存在风险升高的患者设计现实监督计划的能力，通过早期发现这种侵袭性的癌症可能有益于寿命以及生活质量。</p> <p><a href="https://www.nature.com/articles/s41591-023-02332-5" target="_blank" rel="noreferrer">A deep learning algorithm to predict risk of pancreatic cancer from disease trajectories | Nature Medicine</a></p>

2023/6/23
阅读更多

医学人工智能周刊 #3

<h1 id="医学影像分类的自监督学习系统综述以及实施指南" class="relative group">医学影像分类的自监督学习：系统综述以及实施指南 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e5%8c%bb%e5%ad%a6%e5%bd%b1%e5%83%8f%e5%88%86%e7%b1%bb%e7%9a%84%e8%87%aa%e7%9b%91%e7%9d%a3%e5%ad%a6%e4%b9%a0%e7%b3%bb%e7%bb%9f%e7%bb%bc%e8%bf%b0%e4%bb%a5%e5%8f%8a%e5%ae%9e%e6%96%bd%e6%8c%87%e5%8d%97" aria-label="Anchor">#</a></span></h1><p>深度学习和计算机视觉的发展给医学影像分析提供了有前景的解决方案，有潜力提高医疗水平以及患者治疗效果。然而，训练深度学习模型的主流范式需要大量标注的训练数据，这对于医学影像数据管理既耗时又花费巨大。自监督学习有可能从丰富没有标签的医学数据集中学习有用的见解，为发展鲁棒性高的医学影像模型做出巨大贡献。本综述对不同自监督策略进行了描述，并对2012年到2022年间在PubMed、Scopus、ArXiv上发表的使用自监督学习进行医学影像分类对研究进行系统综述。我们综合了前期工作的知识，并且为未来<strong>利用自监督学习建立医学影像分类模型对研究人员提供了实践指南</strong>。</p> <h2 id="自监督常见技术" class="relative group">自监督常见技术 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e8%87%aa%e7%9b%91%e7%9d%a3%e5%b8%b8%e8%a7%81%e6%8a%80%e6%9c%af" aria-label="Anchor">#</a></span></h2><h3 id="内在关系" class="relative group">内在关系 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e5%86%85%e5%9c%a8%e5%85%b3%e7%b3%bb" aria-label="Anchor">#</a></span></h3><p>在一些手工制定的任务上预训练模型，可以利用数据的内部结构，而无需获取额外的标签。例如图像相对关系、预测图像旋转角度。</p> <h3 id="生成模型" class="relative group">生成模型 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e7%94%9f%e6%88%90%e6%a8%a1%e5%9e%8b" aria-label="Anchor">#</a></span></h3><p>生成模型随着传统的自编码器（AE）、变分编码器（VAE）和生成式对抗网络（GANs）出现而变得流行起来，能够学习训练数据的分布，从而重建原始输入或者创建新的合成数据实例。通过使用现成的数据作为目标，生成模型能在不需要显式标签的情况下被训练用于自动学习有用的隐含表征。</p> <h3 id="对比学习" class="relative group">对比学习 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e5%af%b9%e6%af%94%e5%ad%a6%e4%b9%a0" aria-label="Anchor">#</a></span></h3><p>基于转化图像引起的变化不能改变图像语义的假设。针对相同图像不同的数据增强方法组成了所谓的正样本对，相对于该图像其他图片以及增强样本组成了负样本对。优化模型让正样本对在潜空间距离变小并与负样本距离变远。</p> <ul> <li>SimCLR</li> <li>MoCo</li> </ul> <h3 id="自预测" class="relative group">自预测 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e8%87%aa%e9%a2%84%e6%b5%8b" aria-label="Anchor">#</a></span></h3><p>自预测SSL是对部分输入进行掩码或增强，然后用没有变化的部分来重建原始输入。自预测SSL想法来源于自然语言处理领域掩码模型。</p> <h2 id="微调技术" class="relative group">微调技术 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e5%be%ae%e8%b0%83%e6%8a%80%e6%9c%af" aria-label="Anchor">#</a></span></h2><p>主要有两种策略用于微调已被SSL预训练的模型。如果将任意的影像模型都看成编码器和分类器两部分。两种策略能被分为</p> <ul> <li>端到端的微调，所有权重都训练</li> <li>固定编码器提取特征，对分类器进行微调</li> </ul> <h2 id="自监督医学影像实施指南" class="relative group">自监督医学影像实施指南 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e8%87%aa%e7%9b%91%e7%9d%a3%e5%8c%bb%e5%ad%a6%e5%bd%b1%e5%83%8f%e5%ae%9e%e6%96%bd%e6%8c%87%e5%8d%97" aria-label="Anchor">#</a></span></h2><p>需要多种<strong>自监督学习策略相互比较</strong>，现有研究很少进行比较而是有无自监督学习策略比较。</p> <p>在大型自然图像数据集中自监督预训练的模型也可以被利用到医学影像，但由于<strong>医学影像的独特性</strong>，究竟能迁移多少有待研究。</p> <ul> <li>由于医学图像采集的标准化协议和人体解剖学的同质性，医学图像具有很高的类间视觉相似性，即不同类也很相似；</li> <li>在医学成像领域，感兴趣的语义很少是诸如解剖器官之类的对象，而是该器官或组织内是否存在病理异常。许多异常的特征是非常微妙和局部的视觉线索，这些线索可能会由于增强变得模糊或被掩盖；</li> <li>自预测型自监督学习方法所使用的随机掩码（通过移除有疾病或者异常的图像）可能改变医学影像的语义。</li> </ul> <p>在<strong>对比学习形成正样本对</strong>时应该探索更多策略，而不是使用相同图片的不同增强版本，比如通过临床信息的相似性来定义正样本对。</p> <p>在<strong>自监督学习中引入多模态信息</strong>提高下游任务模型性能。</p> <p><a href="https://www.nature.com/articles/s41746-023-00811-0" target="_blank" rel="noreferrer">Self-supervised learning for medical image classification: a systematic review and implementation guidelines | npj Digital Medicine</a></p>

2023/5/20
阅读更多

医学人工智能周刊 #2

<h2 id="大型语言模型在医学和医学研究中的伦理问题" class="relative group">大型语言模型在医学和医学研究中的伦理问题 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e5%a4%a7%e5%9e%8b%e8%af%ad%e8%a8%80%e6%a8%a1%e5%9e%8b%e5%9c%a8%e5%8c%bb%e5%ad%a6%e5%92%8c%e5%8c%bb%e5%ad%a6%e7%a0%94%e7%a9%b6%e4%b8%ad%e7%9a%84%e4%bc%a6%e7%90%86%e9%97%ae%e9%a2%98" aria-label="Anchor">#</a></span></h2><p>大型语言模型（LLM）是一种在大量文本数据中训练的深度学习模型，其目标是生成类似人类响应的新文本。2022年11月30日发布的基于大型语言模型的对话机器人ChatGPT（OpenAI, San Francisco, CA, USA），将大型语言模型推动到公众视野并且让数百万能够使用它进行试验。自从那时起，医学从业者和研究者就开始探索LLM的潜在应用，因为很多医学实践和研究都围绕着大量基于文本的任务，例如展示、发表、记录和报告。使用LLM来帮助和简化这些任务可以节约大量时间，让临床人员和研究者能进行其他工作。目前有许多在不同开发阶段的其他LLMs，包括BioGPT（MIT），LaMDA（Google），Sparrow（Deepmind AI），Pangu Alpha（Huawei），OPT-IML（Meta）以及Megataron Turing MLG（Nvidia）。一些新的变种，例如基于PubMed文献训练后专注于生物医学领域文本生产以及挖掘的BioGPT，可能对未来医学和医学研究有重大的影响。与任何新兴、颠覆式的技术一样，重要的是考虑使用中的伦理并优先考虑符合社会最佳利益的负责任和有益的应用。<strong>本文从偏见、信任、责任人、公平性以及隐私探讨了LLM在医学实践和研究中关键的伦理问题</strong>。</p> <p><a href="https://www.thelancet.com/journals/landig/article/PIIS2589-7500%2823%2900083-3/fulltext" target="_blank" rel="noreferrer">Ethics of large language models in medicine and medical research - The Lancet Digital Health</a></p> <h2 id="对可穿戴设备数据去隐私化是不是给我们一个虚假的安全感系统综述" class="relative group">对可穿戴设备数据去隐私化是不是给我们一个虚假的安全感？系统综述 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e5%af%b9%e5%8f%af%e7%a9%bf%e6%88%b4%e8%ae%be%e5%a4%87%e6%95%b0%e6%8d%ae%e5%8e%bb%e9%9a%90%e7%a7%81%e5%8c%96%e6%98%af%e4%b8%8d%e6%98%af%e7%bb%99%e6%88%91%e4%bb%ac%e4%b8%80%e4%b8%aa%e8%99%9a%e5%81%87%e7%9a%84%e5%ae%89%e5%85%a8%e6%84%9f%e7%b3%bb%e7%bb%9f%e7%bb%bc%e8%bf%b0" aria-label="Anchor">#</a></span></h2><p>可穿戴设备让采集和分享个人数据变得更加容易，这篇系统综述调查来源于可穿戴设备的去隐私化数据是不是足以保护数据集中个体的隐私。通过文献综述，本文总结正确识别率约86%-100%，表明重新标识风险很高。（重新识别通常是将去标识化或者匿名的数据集与有标识化数据的数据集建立连接）而且，<strong>只需要1-300秒记录数据就可以从通常被认为不能产生标识化信息的传感器数据例如ECG进行重新标识化</strong>。该发现提醒重新思考数据分享的方法，在促进研究创新过程中，同时保护个人隐私。</p> <p><a href="https://www.thelancet.com/journals/landig/article/PIIS2589-7500%2822%2900234-5/fulltext?rss=yes" target="_blank" rel="noreferrer">Does deidentification of data from wearable devices give us a false sense of security? A systematic review - The Lancet Digital Health</a></p> <h2 id="特定时间和事件深度学习方法用于心肌灌注成像后个性化风险评估" class="relative group">特定时间和事件深度学习方法用于心肌灌注成像后个性化风险评估 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e7%89%b9%e5%ae%9a%e6%97%b6%e9%97%b4%e5%92%8c%e4%ba%8b%e4%bb%b6%e6%b7%b1%e5%ba%a6%e5%ad%a6%e4%b9%a0%e6%96%b9%e6%b3%95%e7%94%a8%e4%ba%8e%e5%bf%83%e8%82%8c%e7%81%8c%e6%b3%a8%e6%88%90%e5%83%8f%e5%90%8e%e4%b8%aa%e6%80%a7%e5%8c%96%e9%a3%8e%e9%99%a9%e8%af%84%e4%bc%b0" aria-label="Anchor">#</a></span></h2><p>心肌灌注成像（MPI）的标准临床解释已被证实对主要不良心血管事件（MACE）有预后价值。然而，对特定事件类型和时间间隔进行个性化预测具有挑战性。本文建立一个可解释的深度学习模型，该模型能够直接从MPI和15中临床特征中分别预测全因死亡、急性冠状动脉综合症和血管重建的特定时间风险。这种方法能将<strong>个体事件概率表示为时间的函数并且关注特定患者和特定事件的风险解释</strong>，这可能有助于引起人们对可改变风险因素的关注。该模型使用了<code>DeepHit</code>中架构以及修改后的损失函数，输出是一个3*131的2维矩阵分别代表了从0到每30天直到最长随访时间各事件发生的概率。</p> <p><a href="https://www.nature.com/articles/s41746-023-00806-x" target="_blank" rel="noreferrer">Time and event-specific deep learning for personalized risk assessment after cardiac perfusion imaging | npj Digital Medicine</a></p> <h2 id="临床预测算法验证展望" class="relative group">临床预测算法验证展望 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e4%b8%b4%e5%ba%8a%e9%a2%84%e6%b5%8b%e7%ae%97%e6%b3%95%e9%aa%8c%e8%af%81%e5%b1%95%e6%9c%9b" aria-label="Anchor">#</a></span></h2><p>临床预测算法的泛化性与临床实践应用十分相关。本文基于现有文献概述了三种类型的泛化性：时序，地理，和领域泛化，以及其目标、方法、利益相关方。</p> <p> <figure><img src="https://cdn.jsdelivr.net/gh/jmwyf/pichosting@master/generalizability.png" alt="" class="mx-auto my-0 rounded-md" /> </figure> </p> <p><a href="https://www.nature.com/articles/s41746-023-00832-9" target="_blank" rel="noreferrer">Perspectives on validation of clinical predictive algorithms | npj Digital Medicine</a></p>

2023/5/8
阅读更多

医学人工智能周刊 开刊

<h2 id="通用医学人工智能的基础模型" class="relative group">通用医学人工智能的基础模型 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e9%80%9a%e7%94%a8%e5%8c%bb%e5%ad%a6%e4%ba%ba%e5%b7%a5%e6%99%ba%e8%83%bd%e7%9a%84%e5%9f%ba%e7%a1%80%e6%a8%a1%e5%9e%8b" aria-label="Anchor">#</a></span></h2><p>高灵活性、可复用人工智能模型的异常快速发展可能会在医疗领域迎来新的能力。我们提出一种医学人工智能新范式，称之为通用医学人工智能（GMAI）。<strong>GMAI有能力只用少量数据或者非特定任务标签数据来解决各种各样的任务</strong>。通过在大样本，多样性数据中进行自监督学习，GMAI能灵活地解释不同组合的医学模态，包括影像、电子病历、实验室检查结果、基因组学、图表或者是医学文本。模型会产生富有表现力的输出，包括能显示先进医学推理能力的自由文本解释，口头建议或者图像标注。在本文中我们给出了对于GMAI一系列潜在高影响力的应用，并列出了实现这些应用对应的技术和训练所必须的数据集。我们预计GMAI驱动的应用将挑战当前监管和验证医疗AI器械现有的策略，而且将会改变收集大型医学数据的做法。</p> <p><a href="https://www.nature.com/articles/s41586-023-05881-4" target="_blank" rel="noreferrer">Foundation models for generalist medical artificial intelligence | Nature</a></p> <h2 id="商业医学数据集对医学研究和医疗健康算法的影响" class="relative group">商业医学数据集对医学研究和医疗健康算法的影响 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e5%95%86%e4%b8%9a%e5%8c%bb%e5%ad%a6%e6%95%b0%e6%8d%ae%e9%9b%86%e5%af%b9%e5%8c%bb%e5%ad%a6%e7%a0%94%e7%a9%b6%e5%92%8c%e5%8c%bb%e7%96%97%e5%81%a5%e5%ba%b7%e7%ae%97%e6%b3%95%e7%9a%84%e5%bd%b1%e5%93%8d" aria-label="Anchor">#</a></span></h2><p>随着医疗保健行业进入由云数据存储、分布式计算和机器学习驱动的数字健康新时代，医疗保健数据已成为对私人和公共实体具有价值的优质商品。目前无论是来自工业界、学术界还是政府机构的卫生数据收集和分发框架都并不完善，无法让研究人员充分利用下游分析的能力。在这篇医学政策论文中，我们回顾了现有商业医学数据供应商的现状，特别强调了他们数据的来源、数据可重复性、泛化性的挑战以及数据售卖伦理方面的考虑。我们<strong>主张使用可持续的方法来组织管理公开健康数据，从而让全球人口能够被纳入生物医学研究界</strong>。然而，为了全面实施这些方法，关键利益相关者应齐心协力来让医学数据集更容易获得、更全面和更具有代表性，同时平衡好被收集数据个人隐私以及权利。</p> <p><a href="https://www.thelancet.com/journals/landig/article/PIIS2589-7500%2823%2900025-0/fulltext" target="_blank" rel="noreferrer">The impact of commercial health datasets on medical research and health-care algorithms - The Lancet Digital Health</a></p> <h2 id="用于检测早产儿视网膜病变的定制工程和无代码深度学习模型的开发和国际验证回顾性研究" class="relative group">用于检测早产儿视网膜病变的定制工程和无代码深度学习模型的开发和国际验证：回顾性研究 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e7%94%a8%e4%ba%8e%e6%a3%80%e6%b5%8b%e6%97%a9%e4%ba%a7%e5%84%bf%e8%a7%86%e7%bd%91%e8%86%9c%e7%97%85%e5%8f%98%e7%9a%84%e5%ae%9a%e5%88%b6%e5%b7%a5%e7%a8%8b%e5%92%8c%e6%97%a0%e4%bb%a3%e7%a0%81%e6%b7%b1%e5%ba%a6%e5%ad%a6%e4%b9%a0%e6%a8%a1%e5%9e%8b%e7%9a%84%e5%bc%80%e5%8f%91%e5%92%8c%e5%9b%bd%e9%99%85%e9%aa%8c%e8%af%81%e5%9b%9e%e9%a1%be%e6%80%a7%e7%a0%94%e7%a9%b6" aria-label="Anchor">#</a></span></h2><p>早产儿视网膜病变（ROP）是儿童失明的主要原因，由儿科眼科医生通过间隔筛查进行诊断。然而，早产儿生存率的提高加上现有专家的稀缺，引起了人们对这种方法可持续性的担忧。我们的目标是在英国伦敦的种族多元化人群中开发定制、无代码的基于深度学习的分类器，用于附加病变检查（ROP 的标志），并在四个国家和三大洲的种族、地理和社会经济多样化人群中对其进行外部验证。<strong>无代码深度学习不依赖于受过专业培训的数据科学家，因此对资源匮乏的医疗保健环境具有特别的潜在好处</strong>。</p> <p><a href="https://www.thelancet.com/journals/landig/article/PIIS2589-7500%2823%2900050-X/fulltext" target="_blank" rel="noreferrer">Development and international validation of custom-engineered and code-free deep-learning models for detection of plus disease in retinopathy of prematurity: a retrospective study - The Lancet Digital Health</a></p> <h2 id="目前对基于人工智能的医疗设备临床研究是否足够全面以支持全面的健康技术评估系统综述" class="relative group">目前对基于人工智能的医疗设备临床研究是否足够全面，以支持全面的健康技术评估？系统综述 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e7%9b%ae%e5%89%8d%e5%af%b9%e5%9f%ba%e4%ba%8e%e4%ba%ba%e5%b7%a5%e6%99%ba%e8%83%bd%e7%9a%84%e5%8c%bb%e7%96%97%e8%ae%be%e5%a4%87%e4%b8%b4%e5%ba%8a%e7%a0%94%e7%a9%b6%e6%98%af%e5%90%a6%e8%b6%b3%e5%a4%9f%e5%85%a8%e9%9d%a2%e4%bb%a5%e6%94%af%e6%8c%81%e5%85%a8%e9%9d%a2%e7%9a%84%e5%81%a5%e5%ba%b7%e6%8a%80%e6%9c%af%e8%af%84%e4%bc%b0%e7%b3%bb%e7%bb%9f%e7%bb%bc%e8%bf%b0" aria-label="Anchor">#</a></span></h2><p>基于人工智能的医疗设备（AI-based MDs）在医疗保健领域正在经历指数级增长。这项研究旨在调查目前评估人工智能的研究是否包含健康技术评价（HTA）机构进行HTA所需的信息。在讨论部分，从各个角色的角度分析，建议该从哪些方面进行改进AI-MD评价</p> <p><a href="https://www.sciencedirect.com/science/article/pii/S0933365723000611" target="_blank" rel="noreferrer">Are current clinical studies on artificial intelligence-based medical devices comprehensive enough to support a full health technology assessment? A systematic review - ScienceDirect</a></p> <p>#medical-data</p>

2023/5/1
阅读更多

生存分析（一）

<script> document.addEventListener("DOMContentLoaded", function() { renderMathInElement(document.body, { delimiters: [ {left: '$$', right: '$$', display: true}, {left: '$', right: '$', display: false}, {left: '\\(', right: '\\)', display: false}, {left: '\\[', right: '\\]', display: true} ], throwOnError : false }); }); </script> <h2 id="简介" class="relative group">简介 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e7%ae%80%e4%bb%8b" aria-label="Anchor">#</a></span></h2><p>生存分析是将<strong>观察的结局</strong>和<strong>出现结局所经历的时间</strong>结合起来进行分析的一系列统计方法，常用于研究<strong>影响因素</strong>与生存时间和结局的关系，预测不同因素水平<strong>个体生存预测</strong>。</p> <p>因为跟时间相关，所以要定义要<strong>事件起点</strong>，以及<strong>事件终点</strong>。<strong>生存时间</strong>T也可以根据事件起终点计算出来。</p> <p>由于有些事件无法被观测或者没有观察到，导致生存时间无法被记录的情况称为<strong>删失</strong>。其中最为常见的情形称为右删失（right censoring，下图）</p> <ul> <li>右删失：对这样的病人我们只知道其生存时间要大于从试验开始到删失发生的时间。有多种原因可以导致右删失情况的出现，其中包括：(1)病人在某时间点上退出试验或失去随访信息；(2)病人在整个试验结束时事件还未发生；(3)病人由于毒性等原因停用被分派的药物或换用其它药物；(4)竞争风险事件的发生<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup>。 <figure><img src="https://cdn.jsdelivr.net/gh/jmwyf/pichosting@master/Rcensor.jpg" alt="" class="mx-auto my-0 rounded-md" /> </figure> </li> <li>左删失<sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup>：事件发生了，且发生时间在（0，t） ，但确切时间并不清楚。</li> <li>区间删失: 事件发生在一个已知的具体时间段内，但是并不知道具体时间。</li> </ul> <h2 id="生存函数" class="relative group">生存函数 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e7%94%9f%e5%ad%98%e5%87%bd%e6%95%b0" aria-label="Anchor">#</a></span></h2><p>生存函数$$S(t) = P(T&gt;t)$$描述的是<strong>事件发生时间大于时间点t的概率</strong>，理论上T是连续变量，那么生存函数是一个连续的递减函数，然而实际实验中T都是有限时间点的，所以连续曲线变成离散的。“若想用光滑曲线来连接就需要对随机变量T的分布做出假设的参数拟合法，而曲线一般不宜正好经过所有的红点 (那样会导致过度拟合而使得统计模型没有多大效用) ；若用非参数的阶梯函数来连接，那么曲线简单而唯一确定！”这也是为什么k-m曲线这么重要的原因。 <figure><img src="https://cdn.jsdelivr.net/gh/jmwyf/pichosting@master/surivial.jpg" alt="" class="mx-auto my-0 rounded-md" /> </figure> </p> <h2 id="风险函数" class="relative group">风险函数 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e9%a3%8e%e9%99%a9%e5%87%bd%e6%95%b0" aria-label="Anchor">#</a></span></h2><p>风险函数的定义即在<strong>t时刻发生事件的概率</strong>。$$h(t)=P(T=t|T\ge t)$$生存函数是可以看到是从1到0递减的，而风险函数没有固定的单调性，可以是常数风险、可以随时间变化，比如上升、下降、先下降后上升。</p> <p>假设生存时间T这个变量概率分布满足$f(t)$，累积分布$F(t)=P(T\le t)= \int_0^t f(t)dt$，可以看到生存函数$S(t)=1-F(t)$，两边求导数可得$$f(t)=-S&rsquo;(t)$$ 那么在t时刻发生事件的风险用极限的观点来看即在$t -&gt; t+\Delta t$ 这个区间内发生事件的数量除以在t时刻剩下的总人数$$h(t)=\lim_{\Delta t \to 0}\frac{F(t+\Delta t)-F(t)}{\Delta t*S(t)}=\frac{f(t)}{S(t)}=\frac{-S&rsquo;(t)}{S(t)}$$ $$h(t)=\frac{-S&rsquo;(t)}{S(t)}=-\frac{\partial}{\partial y}log[S(t)]\tag1$$</p> <p>$$S(t)=exp[-\int_0^th(t)dt]\tag2$$ 式1后面一个等式右边求偏微分等于等式左边，两边积分即式2。以上为风险函数与生存函数之间的关系<sup id="fnref:3"><a href="#fn:3" class="footnote-ref" role="doc-noteref">3</a></sup></p> <h2 id="kaplan-meier生存曲线" class="relative group">Kaplan-Meier生存曲线 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#kaplan-meier%e7%94%9f%e5%ad%98%e6%9b%b2%e7%ba%bf" aria-label="Anchor">#</a></span></h2><p>K-M曲线是非参数估计生存函数的一种方法。</p> <p>如何绘制K-M生存曲线？ <strong>临床原始资料</strong>一般如下：</p> <table> <thead> <tr> <th>患者</th> <th>生存时间</th> <th>发生事件与否</th> <th>删失与否</th> </tr> </thead> <tbody> <tr> <td>a</td> <td>10</td> <td>0</td> <td>r</td> </tr> <tr> <td>b</td> <td>28</td> <td>1</td> <td></td> </tr> <tr> <td>c</td> <td>30</td> <td>1</td> <td></td> </tr> <tr> <td>d</td> <td>2</td> <td>1</td> <td></td> </tr> <tr> <td>e</td> <td>7</td> <td>0</td> <td>r</td> </tr> <tr> <td>假设是来分析患者生存分析，定义结局是死亡，最长实验观察时间是90天，</td> <td></td> <td></td> <td></td> </tr> </tbody> </table> <p>首先是判断删失数据，假设a和e患者分别出院，观察不到死亡结局，那么以上两个患者是删失数据。</p> <p>将资料整理成<strong>生存概率随时间变化表格</strong></p> <table> <thead> <tr> <th>患者</th> <th>生存时间</th> <th>存活患者数n</th> <th>死亡人数d</th> <th>当前存活概率(n-d)/n</th> <th>生存概率</th> </tr> </thead> <tbody> <tr> <td>0</td> <td></td> <td></td> <td></td> <td></td> <td>1</td> </tr> <tr> <td>d</td> <td>2</td> <td>5</td> <td>1</td> <td>4/5</td> <td>4/5</td> </tr> <tr> <td>e</td> <td>7+</td> <td></td> <td></td> <td></td> <td></td> </tr> <tr> <td>a</td> <td>10+</td> <td></td> <td></td> <td></td> <td></td> </tr> <tr> <td>b</td> <td>28</td> <td>2</td> <td>1</td> <td>1/2</td> <td>${4/5}*{1/2}=2/5$</td> </tr> <tr> <td>c</td> <td>30</td> <td>1</td> <td>1</td> <td>0</td> <td>0</td> </tr> </tbody> </table> <p>将上表生存时间以及生存概率可视化即可得到k-m曲线。</p> <p>以上介绍了生存分析的基本原理以及非参数生存分析K-M曲线是如何来的，有空介绍半参以及参数化的生存分析方法。</p> <div class="footnotes" role="doc-endnotes"> <hr> <ol> <li id="fn:1"> <p><a href="https://m.thepaper.cn/baijiahao_19523218" target="_blank" rel="noreferrer">揭秘生存曲线背后的生物统计学</a>&#160;<a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:2"> <p>怎么理解生存分析的风险函数? - 数据的小米虫的回答 - 知乎 <a href="https://www.zhihu.com/question/343779367/answer/2439383246" target="_blank" rel="noreferrer">https://www.zhihu.com/question/343779367/answer/2439383246</a>&#160;<a href="#fnref:2" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:3"> <p>怎么理解生存分析的风险函数? - 郭老师医学统计的回答 - 知乎 <a href="https://www.zhihu.com/question/343779367/answer/1493205766" target="_blank" rel="noreferrer">https://www.zhihu.com/question/343779367/answer/1493205766</a>&#160;<a href="#fnref:3" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> </ol> </div>

2023/3/30
阅读更多

统计学中的假设检验

<script> document.addEventListener("DOMContentLoaded", function() { renderMathInElement(document.body, { delimiters: [ {left: '$$', right: '$$', display: true}, {left: '$', right: '$', display: false}, {left: '\\(', right: '\\)', display: false}, {left: '\\[', right: '\\]', display: true} ], throwOnError : false }); }); </script> <h2 id="简介" class="relative group">简介 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e7%ae%80%e4%bb%8b" aria-label="Anchor">#</a></span></h2><p>药厂宣传新药疗效很好，研究宣称研发的算法比之前的要好或者某项运动是有助于长寿的，我们怎么样来判断这些结果是否靠谱？这些问题就可以用统计学中的假设检验来判断。</p> <p><strong>统计推断</strong>是根据抽样分布规律和概率理论，由样本结果去推论总体特征。它主要包括假设检验和参数估计两个内容。</p> <p>假设检验的理论依据是“小概率事件原理”。“小概率事件原理”就是概率很小的事件在一次试验中认为是不可能发生的。如果预先的假设使得小概率事件发生了，类似于数学中传统推理的反证法出现逻辑矛盾那样，就认为出现了不合理现象，从而拒绝假设。一般把概率不超过0.10、0.05、0.01的事件当作“小概率事件”，称为检验水准或显著水平，通常取0.05、0.01，实际问题中也可取0.10、0.001等。<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup></p> <h2 id="假设检验步骤" class="relative group">假设检验步骤 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e5%81%87%e8%ae%be%e6%a3%80%e9%aa%8c%e6%ad%a5%e9%aa%a4" aria-label="Anchor">#</a></span></h2><h3 id="提出假设原假设和备择假设" class="relative group">提出假设原假设和备择假设 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e6%8f%90%e5%87%ba%e5%81%87%e8%ae%be%e5%8e%9f%e5%81%87%e8%ae%be%e5%92%8c%e5%a4%87%e6%8b%a9%e5%81%87%e8%ae%be" aria-label="Anchor">#</a></span></h3><ul> <li>根据要比较的统计量类型，选择不同的假设检验类型，比如样本均值与指定值，汽车百公里油耗为xx；样本比例，支持率低于30%；样本方差，矿泉水容量的离散程度</li> <li>原假设通常是不存在差异或者没有关联，比如A组和B组均值没有差异；备选假设可以选择左右或双侧（大、小或不等于） <ul> <li>原假设是唯一的，而备择假设有很多，这也是为什么对无差异进行检验的原因，即<strong>假设无差异为真来检验到底是不是无差异</strong></li> </ul> </li> </ul> <h3 id="抽样optional" class="relative group">抽样（optional） <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e6%8a%bd%e6%a0%b7optional" aria-label="Anchor">#</a></span></h3><ul> <li>一般问题就说了自己样本是什么。对于需要实验验证的问题，采样时由于不可能涵盖所有的样本，需要选择合适具有代表性的样本，进行两组比较或者与指定总体样本比较</li> </ul> <h3 id="选择检验统计量" class="relative group">选择检验统计量 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e9%80%89%e6%8b%a9%e6%a3%80%e9%aa%8c%e7%bb%9f%e8%ae%a1%e9%87%8f" aria-label="Anchor">#</a></span></h3><ul> <li>对假设进行检验的统计量，一般为抽样的样本在原假设情况下符合什么分布，计算对所关注差异或者效果的度量。</li> </ul> <h3 id="统计显著性水平拒绝域临界点" class="relative group">统计显著性水平、拒绝域/临界点 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e7%bb%9f%e8%ae%a1%e6%98%be%e8%91%97%e6%80%a7%e6%b0%b4%e5%b9%b3%e6%8b%92%e7%bb%9d%e5%9f%9f%e4%b8%b4%e7%95%8c%e7%82%b9" aria-label="Anchor">#</a></span></h3><ul> <li><strong>显著性水平</strong>：原假设为真时拒绝原假设的概率，一般是0.05。后续计算以原假设为真统计量出现的概率，假如小于显著性水平，即认为出现的概率很低（小概率事件），拒绝原假设。<strong>假如原假设确实为真，这里就犯了第一类错误，但是这个概率是我们可以直接设置的。</strong> <ul> <li><strong>第一类错误</strong><sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup>：没差异判断有差异$\alpha$</li> <li><strong>第二类错误</strong>：有差异判断没差异$\beta$</li> <li>统计检验能力：有差异能判断出差异，$1-\beta$ 通常需要大雨0.8（Jacob Cohen）</li> <li>第一类错误和第二类错误的关系 <ul> <li>如下图应该选择备择假设，而选了null假设的概率为$\beta$, 可以看到$\alpha$越小，$\beta$越大,无法设置让第一类错误和第二类错误同时变小</li> <li> <figure><img src="https://cdn.jsdelivr.net/gh/jmwyf/pichosting@master/error1.png" alt="" class="mx-auto my-0 rounded-md" /> </figure> </li> </ul> </li> </ul> </li> <li>拒绝域：根据分布以及显著性水平可以确定拒绝域值</li> </ul> <h3 id="求出检查统计量的p值" class="relative group">求出检查统计量的p值 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e6%b1%82%e5%87%ba%e6%a3%80%e6%9f%a5%e7%bb%9f%e8%ae%a1%e9%87%8f%e7%9a%84p%e5%80%bc" aria-label="Anchor">#</a></span></h3><ul> <li>双边或单边某中分布下检验量统计量出现的概率</li> </ul> <h3 id="查看样本结果是否位于拒绝域" class="relative group">查看样本结果是否位于拒绝域 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e6%9f%a5%e7%9c%8b%e6%a0%b7%e6%9c%ac%e7%bb%93%e6%9e%9c%e6%98%af%e5%90%a6%e4%bd%8d%e4%ba%8e%e6%8b%92%e7%bb%9d%e5%9f%9f" aria-label="Anchor">#</a></span></h3><ul> <li>一般通过p与显著性水平比较</li> </ul> <h3 id="决定是否接受原假设" class="relative group">决定是否接受原假设 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e5%86%b3%e5%ae%9a%e6%98%af%e5%90%a6%e6%8e%a5%e5%8f%97%e5%8e%9f%e5%81%87%e8%ae%be" aria-label="Anchor">#</a></span></h3><ul> <li>如果p小于显著性水平，拒绝原假设</li> </ul> <h2 id="案例一-女士品茶" class="relative group">案例一 女士品茶 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e6%a1%88%e4%be%8b%e4%b8%80-%e5%a5%b3%e5%a3%ab%e5%93%81%e8%8c%b6" aria-label="Anchor">#</a></span></h2><p>以女士品茶为例，一位女士宣称自己可以分辨先奶后茶还是先茶后奶，大家都觉得不可思议，准备了10杯奶茶让其分辨，正确分辨了每一杯，现在问题是是不是真的能分辨？</p> <ul> <li>提出原假设和备择假设 <ul> <li>原假设：不能分辨</li> </ul> </li> <li>选择检验统计量 <ul> <li>在不能分辨的情况10次都对，对该事件的度量</li> </ul> </li> <li>显著性水平 <ul> <li>0.05</li> </ul> </li> <li>检验统计量概率：不能分辨就是瞎猜每次判断的概率为1/2 ，该次事件的概率为$(\frac{1}{2})^{10}$</li> <li>判断是否位于拒绝域 <ul> <li>远小于显著性水平，落在拒绝域，小概率事件不可能发生，但是在这次实验中发生，说明原假设不对</li> </ul> </li> <li>拒绝原假设 <ul> <li>即该女士有分辨奶茶先后的能力</li> </ul> </li> </ul> <h2 id="案例二-均值检验" class="relative group">案例二 均值检验 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e6%a1%88%e4%be%8b%e4%ba%8c-%e5%9d%87%e5%80%bc%e6%a3%80%e9%aa%8c" aria-label="Anchor">#</a></span></h2><p>测定土地PH值是否为7，进行了17次采样，采样结果mean = 6.676， sd= 0.455<sup id="fnref:3"><a href="#fn:3" class="footnote-ref" role="doc-noteref">3</a></sup></p> <ul> <li>提出原假设与备择假设 <ul> <li>ph为7</li> </ul> </li> <li>选择检验统计量 <ul> <li>大样本数据一般认为符合正态分布，正态分布的均值$$z=\frac{\bar{x}-\mu_{0}}{\sigma/\sqrt{n}}$$而总体标准差一般很难获取，这时一般用样本标准差来代替，样本统计量服从t分布，上述公式变为$$z=\frac{\bar{x}-\mu_{0}}{s/\sqrt{n}}$$样本的均值应满足上式</li> </ul> </li> <li>显著性水平 <ul> <li>0.05</li> </ul> </li> <li>检验统计量概率 <ul> <li>代入公式得t统计量为-2.9，查分布表找到概率为0.009</li> </ul> </li> <li>拒绝</li> </ul> <div class="footnotes" role="doc-endnotes"> <hr> <ol> <li id="fn:1"> <p>实验设计与统计分析&#160;<a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:2"> <p>统计学图鉴&#160;<a href="#fnref:2" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:3"> <p>统计学习理论与方法&#160;<a href="#fnref:3" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> </ol> </div>

2023/3/22
阅读更多

什么是科学问题？

<p>科学问题：地球人还未找到答案，且无法确定是否能找到答案的问题 ；<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup><sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup></p> <p>技术问题：实现某个现实已有功能的途径；</p> <p>工程问题：结合多个技术解决一个系统化现实问题的方案；</p> <p>（科学问题：我不知道；</p> <p>技术问题：我没想到；</p> <p>工程问题：我做不了；）</p> <p>参考新版世界前沿125个科学问题<sup id="fnref:3"><a href="#fn:3" class="footnote-ref" role="doc-noteref">3</a></sup></p> <h2 id="sjtu--science-125个科学问题2021年版本" class="relative group"><strong>SJTU &amp; Science 125个科学问题</strong>（2021年版本） <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#sjtu--science-125%e4%b8%aa%e7%a7%91%e5%ad%a6%e9%97%ae%e9%a2%982021%e5%b9%b4%e7%89%88%e6%9c%ac" aria-label="Anchor">#</a></span></h2><h2 id="medicine--health-11" class="relative group"><strong>Medicine &amp; Health</strong> (11) <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#medicine--health-11" aria-label="Anchor">#</a></span></h2><p><strong>医学与健康</strong></p> <ol> <li>Can we predict the next pandemic?</li> </ol> <p>1.我们可以预测下一次流行病吗？</p> <ol start="2"> <li>Will we ever find a cure for the common cold?</li> </ol> <p>2.我们会找到治疗感冒的方法吗？</p> <ol start="3"> <li>Can we design and manufacture medicines customized for individual people?</li> </ol> <p>3.我们可以设计和制造出为个人定制的药物吗？</p> <ol start="4"> <li>Can a human tissue or organ be fully regenerated?</li> </ol> <p>4.人体组织或器官可以完全再生吗？</p> <ol start="5"> <li>How is immune homeostasis maintained and regulated?</li> </ol> <p>5.如何维持和调节免疫稳态？</p> <ol start="6"> <li>Is there a scientific basis to the Meridian System in traditional Chinese medicine?</li> </ol> <p>6.中医的经络系统有科学依据吗？</p> <ol start="7"> <li>How will the next generation of vaccines be made?</li> </ol> <p>7.下一代疫苗将如何生产？</p> <ol start="8"> <li>Can we ever overcome antibiotic resistance?</li> </ol> <p>8.我们能否克服抗生素耐药性？</p> <ol start="9"> <li>What is the etiology of autism?</li> </ol> <p>9.自闭症的病因是什么？</p> <ol start="10"> <li>What role does our microbiome play in health and disease?</li> </ol> <p>10.我们的微生物组在健康和疾病中扮演什么角色？</p> <ol start="11"> <li>Can xenotransplantation solve the shortage of donor organs?</li> </ol> <p>11.异种移植能否解决供体器官的短缺问题？</p> <h2 id="biology22" class="relative group"><strong>Biology</strong>(22) <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#biology22" aria-label="Anchor">#</a></span></h2><p><strong>生命科学</strong></p> <ol> <li>What could help conservation of the oceans?</li> </ol> <p>1.什么可以帮助保护海洋？</p> <ol start="2"> <li>Can we stop ourselves from aging?</li> </ol> <p>2.我们可以阻止自己衰老吗？</p> <ol start="3"> <li>Why can only some cells become other cells?</li> </ol> <p>3.为什么只有一些细胞会变成其他细胞？</p> <ol start="4"> <li>Why are some genomes so big and others very small?</li> </ol> <p>4.为什么有些基因组非常大而另一些却很小？</p> <ol start="5"> <li>Will it be possible to cure all cancers?</li> </ol> <p>5.有可能治愈所有癌症吗？</p> <ol start="6"> <li>What genes make us uniquely human?</li> </ol> <p>6.哪些基因使我们人类与众不同？</p> <ol start="7"> <li>How do migratory animals know where they&rsquo;re going?</li> </ol> <p>7.迁徙动物如何知道它们要去哪里？</p> <ol start="8"> <li>How many species are there on Earth?</li> </ol> <p>8.地球上有多少物种？</p> <ol start="9"> <li>How do organisms evolve?</li> </ol> <p>9.有机体是如何进化的？</p> <ol start="10"> <li>Why did dinosaurs grow to be so big?</li> </ol> <p>10.为什么恐龙长得如此之大？</p> <ol start="11"> <li>Did ancient humans interbreed with other human-like ancestors?</li> </ol> <p>11.远古人类是否曾与其他类人祖先杂交？</p> <ol start="12"> <li>Why do humans get so attached to dogs and cats?</li> </ol> <p>12.人类为什么会对猫狗如此着迷？</p> <ol start="13"> <li>Will the world&rsquo;s population keep growing indefinitely?</li> </ol> <p>13.世界人口会无限增长吗？</p> <ol start="14"> <li>Why do we stop growing?</li> </ol> <p>14.我们为什么会停止生长？</p> <ol start="15"> <li>Is de-extinction possible?</li> </ol> <p>15.能否复活灭绝生物？</p> <ol start="16"> <li>Can humans hibernate?</li> </ol> <p>16.人类可以冬眠吗？</p> <ol start="17"> <li>Where do human emotions originate?</li> </ol> <p>17.人类的情感源于何处？</p> <ol start="18"> <li>Will humans look physically different in the future?</li> </ol> <p>18.未来人类的外貌会有所不同吗？</p> <ol start="19"> <li>Why were there species explosions and mass extinction?</li> </ol> <p>19.为什么会发生物种大爆发和大灭绝？</p> <ol start="20"> <li>How might genome editing be used to cure disease?</li> </ol> <p>20.基因组编辑将如何用于治疗疾病？</p> <ol start="21"> <li>Can a cell be artificially synthesized?</li> </ol> <p>21.可以人工合成细胞吗？</p> <ol start="22"> <li>How are biomolecules organized in cells to function orderly and effectively?</li> </ol> <p>22.细胞内的生物分子是如何组织从而有序有效发挥作用的？</p> <h2 id="information-science4" class="relative group"><strong>Information Science</strong>(4) <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#information-science4" aria-label="Anchor">#</a></span></h2><p><strong>信息科学</strong></p> <ol> <li>Is there an upper limit to computer processing speed?</li> </ol> <p>1.计算机处理速度是否有上限？</p> <ol start="2"> <li>Can AI replace a doctor?</li> </ol> <p>2.AI可以代替医生吗？</p> <ol start="3"> <li>Can topological quantum computing be realized?</li> </ol> <p>3.拓扑量子计算可以实现吗？</p> <ol start="4"> <li>Can DNA act as an information storage medium?</li> </ol> <p>4.DNA可以用作信息存储介质吗？</p> <h2 id="neuroscience12" class="relative group"><strong>Neuroscience</strong>(12) <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#neuroscience12" aria-label="Anchor">#</a></span></h2><p><strong>神经科学</strong></p> <ol> <li>What are the coding principles embedded in neuronal spike trains?</li> </ol> <p>1.神经元放电序列的编码准则是什么？</p> <ol start="2"> <li>Where does consciousness lie?</li> </ol> <p>2.意识存在于何处？</p> <ol start="3"> <li>Can human memory be stored, manipulated, and transplanted digitally?</li> </ol> <p>3.能否数字化地存储、操控和移植人类记忆？</p> <ol start="4"> <li>Why do we need sleep?</li> </ol> <p>4.为什么我们需要睡眠？</p> <ol start="5"> <li>What is addiction and how does it work?</li> </ol> <p>5.什么是成瘾？</p> <ol start="6"> <li>Why do we fall in love?</li> </ol> <p>6.为什么我们会坠入爱河？</p> <ol start="7"> <li>How did speech evolve and what parts of the brain control it?</li> </ol> <p>7.言语如何演变形成，大脑的哪些部分对其进行控制？</p> <ol start="8"> <li>How smart are nonhuman animals?</li> </ol> <p>8.除人类以外的其他动物有多聪明？</p> <ol start="9"> <li>Why are most people right-handed?</li> </ol> <p>9.为什么大多数人都是右撇子？</p> <ol start="10"> <li>Can we cure neurodegenerative diseases?</li> </ol> <p>10.我们可以治愈神经退行性疾病吗？</p> <ol start="11"> <li>Is it possible to predict the future?</li> </ol> <p>11.有可能预知未来吗？</p> <ol start="12"> <li>Can we more effectively diagnose and treat complex mental disorders?</li> </ol> <p>12.精神障碍能否有效诊断和治疗？</p> <h2 id="artificial-intelligence8" class="relative group"><strong>Artificial Intelligence</strong>(8) <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#artificial-intelligence8" aria-label="Anchor">#</a></span></h2><p><strong>人工智能</strong></p> <ol> <li>Will injectable, disease-fighting nanobots ever be a reality?</li> </ol> <p>1.可注射的抗病纳米机器人会成为现实吗？</p> <ol start="2"> <li>Will it be possible to create sentient robots?</li> </ol> <p>2.是否有可能创建有感知力的机器人？</p> <ol start="3"> <li>Is there a limit to human intelligence?</li> </ol> <p>3.人类智力是否有极限？</p> <ol start="4"> <li>Will artificial intelligence replace humans?</li> </ol> <p>4.人工智能会取代人类吗？</p> <ol start="5"> <li>How does group intelligence emerge?</li> </ol> <p>5.群体智能是如何出现的？</p> <ol start="6"> <li>Can robots or AIs have human creativity?</li> </ol> <p>6.机器人或 AI 可以具有人类创造力吗？</p> <ol start="7"> <li>Can quantum artificial intelligence imitate the human brain?</li> </ol> <p>7.量子人工智能可以模仿人脑吗？</p> <ol start="8"> <li>Could we integrate with computers to form a human-machine hybrid species?</li> </ol> <p>8.我们可以和计算机结合以形成人机混合物种吗？</p> <h2 id="mathematical-sciences3" class="relative group"><strong>Mathematical Sciences</strong>(3) <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#mathematical-sciences3" aria-label="Anchor">#</a></span></h2><p><strong>数学</strong></p> <ol> <li>What makes prime numbers so special?</li> </ol> <p>1.什么使素数如此特别？</p> <ol start="2"> <li>Will the Navier–Stokes problem ever be solved?</li> </ol> <p>2.纳维尔-斯托克斯问题会得到解决吗？</p> <ol start="3"> <li>Is the Riemann hypothesis true?</li> </ol> <p>3.黎曼猜想是真的吗？</p> <h2 id="chemistry9" class="relative group"><strong>Chemistry</strong>(9) <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#chemistry9" aria-label="Anchor">#</a></span></h2><p><strong>化学</strong></p> <ol> <li>Are there more color pigments to discover?</li> </ol> <p>1.还有更多色彩元素可发现吗？</p> <ol start="2"> <li>Will the periodic table ever be complete?</li> </ol> <p>2.元素周期表会完整吗？</p> <ol start="3"> <li>How can we measure interface phenomena on the microscopic level?</li> </ol> <p>3.如何在微观层面测量界面现象？</p> <ol start="4"> <li>What is the future for energy storage？</li> </ol> <p>4.能量存储的未来是怎样的？</p> <ol start="5"> <li>Why does life require chirality?</li> </ol> <p>5.为什么生命需要手性？</p> <ol start="6"> <li>How can we better manage the world&rsquo;s plastic waste?</li> </ol> <p>6.我们如何更好地管理世界上的塑料废物？</p> <ol start="7"> <li>Will AI redefine the future of chemistry?</li> </ol> <p>7.AI会重新定义化学的未来吗？</p> <ol start="8"> <li>How can matter be programmed into living materials?</li> </ol> <p>8.物质如何被编码而成为生命材料？</p> <ol start="9"> <li>What drives reproduction in living systems?</li> </ol> <p>9.是什么驱动生命系统的复制？</p> <h2 id="astronomy23" class="relative group"><strong>Astronomy</strong>(23) <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#astronomy23" aria-label="Anchor">#</a></span></h2><p><strong>天文学</strong></p> <ol> <li>How many dimensions are there in space?</li> </ol> <p>1.空间中有多少个维度？</p> <ol start="2"> <li>What is the shape of the universe?</li> </ol> <p>2.宇宙的形状是怎样的？</p> <ol start="3"> <li>Where did the big bang start?</li> </ol> <p>3.大爆炸从何处开始？</p> <ol start="4"> <li>Why don&rsquo;t the orbits of planets decay and cause them to crash into each other?</li> </ol> <p>4.为什么行星的轨道不衰减并导致它们相互碰撞？</p> <ol start="5"> <li>When will the universe die? Will it continue to expand?</li> </ol> <p>5.宇宙何时消亡？它会继续膨胀吗？</p> <ol start="6"> <li>Is it possible to live permanently on another planet?</li> </ol> <p>6.我们有可能在另一个星球上长期居住吗？</p> <ol start="7"> <li>Why do black holes exist?</li> </ol> <p>7.为什么存在黑洞？</p> <ol start="8"> <li>What is the universe made of?</li> </ol> <p>8.宇宙是由什么构成的？</p> <ol start="9"> <li>Are we alone in the universe?</li> </ol> <p>9.我们是宇宙中唯一的生命体吗？</p> <ol start="10"> <li>What is the origin of cosmic rays?</li> </ol> <p>10.宇宙射线的起源是什么？</p> <ol start="11"> <li>What is the origin of mass?</li> </ol> <p>11.物质的起源是什么？</p> <ol start="12"> <li>What is the smallest scale of space-time?</li> </ol> <p>12.时空的最小尺度是是多少？</p> <ol start="13"> <li>Is water necessary for all life in the universe, or just on Earth?</li> </ol> <p>13.水是宇宙中所有生命所必需的么，还是仅对地球生命？</p> <ol start="14"> <li>What is preventing humans from carrying out deep-space exploration?</li> </ol> <p>14.是什么阻止了人类进行深空探测？</p> <ol start="15"> <li>Is Einstein&rsquo;s general theory of relativity correct?</li> </ol> <p>15.爱因斯坦的广义相对论是正确的吗？</p> <ol start="16"> <li>How are pulsars formed?</li> </ol> <p>16.脉冲星是如何形成的？</p> <ol start="17"> <li>Is our Milky Way Galaxy special?</li> </ol> <p>17.我们的银河系特别吗？</p> <ol start="18"> <li>What is the volume, composition, and significance of the deep biosphere?</li> </ol> <p>18.深层生物圈的规模、组成和意义是什么？</p> <ol start="19"> <li>Will humans one day have to leave the planet (or die trying)?</li> </ol> <p>19.人类有一天会不得不离开地球吗（还是会在尝试中死去）？</p> <ol start="20"> <li>Where do the heavy elements in the universe come from?</li> </ol> <p>20.宇宙中的重元素来自何处？</p> <ol start="21"> <li>Is it possible to understand the structure of compact stars and matter?</li> </ol> <p>21.有可能了解致密恒星和物质的结构吗？</p> <ol start="22"> <li>What is the origin of the high-energy cosmic neutrinos?</li> </ol> <p>22.高能宇宙中微子的起源是什么？</p> <ol start="23"> <li>What is gravity?</li> </ol> <p>23.什么是重力？</p> <h2 id="physics18" class="relative group"><strong>Physics</strong>(18) <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#physics18" aria-label="Anchor">#</a></span></h2><p><strong>物理学</strong></p> <ol> <li>Is there a diffraction limit?</li> </ol> <p>1.有衍射极限吗？</p> <ol start="2"> <li>What is the microscopic mechanism for high-temperature superconductivity?</li> </ol> <p>2.高温超导的微观机理是什么？</p> <ol start="3"> <li>What are the limits of heat transfer in matter?</li> </ol> <p>3.物质传热的极限是什么？</p> <ol start="4"> <li>What are the fundamental principles of collective motion?</li> </ol> <p>4.集体运动的基本原理是什么？</p> <ol start="5"> <li>What are the smallest building blocks of matter?</li> </ol> <p>5.什么是物质的最小组成部分？</p> <ol start="6"> <li>Will we ever travel at the speed of light?</li> </ol> <p>6.我们会以光速行驶吗？</p> <ol start="7"> <li>What is quantum uncertainty and why is it important?</li> </ol> <p>7.什么是量子不确定性，为什么它很重要？</p> <ol start="8"> <li>Will there ever be a &ldquo;theory of everything&rdquo;?</li> </ol> <p>8.会有“万有理论”吗？</p> <ol start="9"> <li>Why does time seem to flow in only one direction?</li> </ol> <p>9.为什么时间似乎只朝一个方向流动？</p> <ol start="10"> <li>What is dark matter?</li> </ol> <p>10.什么是暗物质？</p> <ol start="11"> <li>Can we make a real, human-size invisibility cloak?</li> </ol> <p>11.我们可以制作出真人大小的隐形斗篷吗？</p> <ol start="12"> <li>Are there any particles that behave oppositely to the properties or states of photons?</li> </ol> <p>12.是否存在与光子性质或状态相反的粒子？</p> <ol start="13"> <li>Will the Bose-Einstein condensate be widely used in the future?</li> </ol> <p>13.玻色-爱因斯坦冷凝体未来会被广泛使用吗？</p> <ol start="14"> <li>Can humans make intense lasers with incoherence comparable to sunlight?</li> </ol> <p>14.人类能制造出与太阳光相似的非相干强激光吗？</p> <ol start="15"> <li>What is the maximum speed to which we can accelerate a particle?</li> </ol> <p>15.我们最多可以将粒子加速到多快？</p> <ol start="16"> <li>Is quantum many-body entanglement more fundamental than quantum fields?</li> </ol> <p>16.量子多体纠缠比量子场更基本吗？</p> <ol start="17"> <li>What is the optimum hardware for quantum computers?</li> </ol> <p>17.量子计算机的最佳硬件是什么？</p> <ol start="18"> <li>Can we accurately simulate the macro- and microworld?</li> </ol> <p>18.我们可以精确模拟宏观和微观世界吗？</p> <h2 id="engineering--material-science4" class="relative group"><strong>Engineering &amp; Material Science</strong>(4) <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#engineering--material-science4" aria-label="Anchor">#</a></span></h2><p><strong>工程与材料科学</strong></p> <ol> <li>What is the ultimate statistical invariances of turbulence?</li> </ol> <p>1.湍流的最终统计不变性是什么？</p> <ol start="2"> <li>How can we break the current limit of energy conversion efficiencies?</li> </ol> <p>2.我们如何突破当前的能量转换效率极限？</p> <ol start="3"> <li>How can we develop manufacturing systems on Mars?</li> </ol> <p>3.我们如何在火星上开发制造系统？</p> <ol start="4"> <li>Is a future of only self-driving cars realistic?</li> </ol> <p>4.纯无人驾驶汽车的未来是否现实？</p> <h2 id="ecology8" class="relative group"><strong>Ecology</strong>(8) <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#ecology8" aria-label="Anchor">#</a></span></h2><p><strong>生态学</strong></p> <ol> <li>Can we stop global climate change?</li> </ol> <p>1.我们可以阻止全球气候变化吗？</p> <ol start="2"> <li>Where do we put all the excess carbon dioxide?</li> </ol> <p>2.我们能把过量的二氧化碳存到何处？</p> <ol start="3"> <li>What creates the Earth&rsquo;s magnetic field (and why does it move)?</li> </ol> <p>3.是什么创造了地球的磁场（为什么它会移动）？</p> <ol start="4"> <li>Will we be able to predict catastrophic weather events (tsunami, hurricanes, earthquakes) more accurately?</li> </ol> <p>4.我们是否能够更准确地预测灾害性事件（海啸、飓风、地震）？</p> <ol start="5"> <li>What happens if all the ice on the planet melts?</li> </ol> <p>5.如果地球上所有的冰融化会怎样？</p> <ol start="6"> <li>Can we create an environmentally friendly replacement for plastics?</li> </ol> <p>6.我们可以创造一种环保的塑料替代品吗？</p> <ol start="7"> <li>Can we achieve a situation where essentially every material can be recycled and reused?</li> </ol> <p>7.几乎所有材料都可以回收再利用是否可以实现？</p> <ol start="8"> <li>Will we soon see the end of monocultures like wheat, maize, rice, and soy?</li> </ol> <p>8.我们会很快看到小麦、玉米、大米和大豆等单一作物的终结吗？</p> <h2 id="energy-science3" class="relative group"><strong>Energy Science</strong>(3) <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#energy-science3" aria-label="Anchor">#</a></span></h2><p><strong>能源科学</strong></p> <ol> <li>Could we live in a fossil-fuel-free world?</li> </ol> <p>1.我们可以生活在一个去化石燃料的世界中吗？</p> <ol start="2"> <li>What is the future of hydrogen energy?</li> </ol> <p>2.氢能的未来是怎样的？</p> <ol start="3"> <li>Will cold fusion ever be possible?</li> </ol> <p>3.冷聚变有可能实现吗？</p> <h2 id="思考" class="relative group">思考 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e6%80%9d%e8%80%83" aria-label="Anchor">#</a></span></h2><p>科学问题有大有小，上面是全人类目前这个阶段关注的大科学问题，那么小而精的科学问题该如何凝练？</p> <div class="footnotes" role="doc-endnotes"> <hr> <ol> <li id="fn:1"> <p><a href="https://www.bilibili.com/video/av887401991/" target="_blank" rel="noreferrer">【复旦赵斌】避免尴尬，教你一句话说清楚什么是科学问题、技术问题和工程问题_哔哩哔哩_bilibili</a>&#160;<a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:2"> <p><a href="https://www.bilibili.com/video/BV1oh411q7op/?spm_id_from=pageDriver" target="_blank" rel="noreferrer">【复旦赵斌】老师喊你搞事情 | 125个科学问题，你最想了解哪个？_哔哩哔哩_bilibili</a>c&#160;<a href="#fnref:2" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:3"> <p><a href="https://mp.weixin.qq.com/s/HrV3AVJ5PqdUZVzUPxCxQw" target="_blank" rel="noreferrer">全世界最前沿的125个科学问题！</a>&#160;<a href="#fnref:3" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> </ol> </div>

2023/3/6
阅读更多

MultiBench多模态表征学习的多尺度基准

<p>MULTIBENCH，一个系统而统一的大规模多模态学习基准，涵盖15个数据集、10种模式、20个预测任务和6个研究领域<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup>。</p> <h2 id="引言" class="relative group">引言 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e5%bc%95%e8%a8%80" aria-label="Anchor">#</a></span></h2><p>背景：</p> <ul> <li>语言和视觉领域多模态学习发展不错，但是其他领域欠缺</li> <li>现在的基准评价关注性能，没有量化缺点包括时间空间复杂度，由于不完美模态导致的鲁棒性降低，需要在性能、鲁棒性、复杂度取得平衡</li> </ul> <p>提出multibench就是解决以上问题：</p> <ul> <li>扩充收集各领域数据集、数据模态</li> <li>量化复杂度</li> <li>提出标准流程评价对噪声和缺失模态情况下的鲁棒性</li> </ul> <p>MultiBench是一个端到端的过程，包括数据预处理、数据集拆分、多模态算法、评估指标和交叉验证。</p> <ul> <li>开发工具包MultiZoo</li> <li>可以用于workshop、教学等</li> </ul> <h2 id="多尺度多模态基准" class="relative group">多尺度多模态基准 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e5%a4%9a%e5%b0%ba%e5%ba%a6%e5%a4%9a%e6%a8%a1%e6%80%81%e5%9f%ba%e5%87%86" aria-label="Anchor">#</a></span></h2><p>第一版集中在多模态融合，对于多模态翻译等问题未来版本可能涉及</p> <h3 id="数据集" class="relative group">数据集 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e6%95%b0%e6%8d%ae%e9%9b%86" aria-label="Anchor">#</a></span></h3><p>介绍了6大领域15个数据集，表1</p> <ul> <li>情感计算（affective computing）</li> <li>医疗：时变和静态变量的整合使用</li> <li>机器人</li> <li>金融</li> <li>人机交互</li> <li>多媒体 <figure><img src="https://cdn.jsdelivr.net/gh/jmwyf/pichosting@master/multibenchdata.png" alt="" class="mx-auto my-0 rounded-md" /> </figure> </li> </ul> <h3 id="评价标准" class="relative group">评价标准 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e8%af%84%e4%bb%b7%e6%a0%87%e5%87%86" aria-label="Anchor">#</a></span></h3><p>性能：</p> <ul> <li>regression: MSE, MAE,</li> <li>classification: F1-score, AUPRC</li> </ul> <p>复杂度：</p> <ul> <li>data size in bits</li> <li>number of model parameters</li> <li>time and memory resources on CPU and GPU</li> </ul> <p>鲁棒性：</p> <ul> <li>单模态独有噪音：对图像、音频等单独处理</li> <li>考虑多模态整体的不完善：比如缺失模态等</li> </ul> <h2 id="multizoo多模态算法集合" class="relative group">MultiZoo：多模态算法集合 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#multizoo%e5%a4%9a%e6%a8%a1%e6%80%81%e7%ae%97%e6%b3%95%e9%9b%86%e5%90%88" aria-label="Anchor">#</a></span></h2><p>涵盖实现multibench整个过程中的算法</p> <h3 id="数据预处理" class="relative group">数据预处理 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e6%95%b0%e6%8d%ae%e9%a2%84%e5%a4%84%e7%90%86" aria-label="Anchor">#</a></span></h3><ul> <li>WordAlign算法 <ul> <li>将各模态信息调整到统一粒度</li> </ul> </li> </ul> <h3 id="融合范式" class="relative group">融合范式 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e8%9e%8d%e5%90%88%e8%8c%83%e5%bc%8f" aria-label="Anchor">#</a></span></h3><ul> <li>早期和晚期融合 <ul> <li>EF，LF</li> </ul> </li> <li>多模态张量: 多模态互补 <ul> <li>Tensor Fusion</li> <li>Low-rank Tensor Fusion</li> </ul> </li> <li>多模态乘法交互: 多模态交互 <ul> <li>MI-MATRIX</li> <li>MI-VECTOR</li> <li>MI-SCALAR</li> </ul> </li> <li>多模态门控 <ul> <li>NL GATE: 自注意力机制</li> </ul> </li> <li>时序注意力模型 <ul> <li>MULT: 多模态Transformer</li> </ul> </li> <li>网络架构搜索 <ul> <li>MFAS</li> </ul> </li> </ul> <h3 id="优化目标" class="relative group">优化目标 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e4%bc%98%e5%8c%96%e7%9b%ae%e6%a0%87" aria-label="Anchor">#</a></span></h3><p>除了标准的监督损失函数，纳入一些新提出的目标函数</p> <ul> <li>CCA</li> <li>REFNET</li> <li>MFM</li> <li>MCTN</li> </ul> <h3 id="训练过程" class="relative group">训练过程 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e8%ae%ad%e7%bb%83%e8%bf%87%e7%a8%8b" aria-label="Anchor">#</a></span></h3><ul> <li>Gradient Blending来计算融合的权重</li> <li>Regularization by Maximizing Functional Entropies</li> </ul> <h2 id="实验" class="relative group">实验 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e5%ae%9e%e9%aa%8c" aria-label="Anchor">#</a></span></h2><ul> <li>泛化性能 <ul> <li>目前的方法表现出高方差，没有放之四海而皆准的模型，特别是对于未被研究的模式和任务。</li> <li>后期融合表现比较均衡</li> <li>有些融合方法是专门为2模态设计，有些在2/3模态表现不好</li> </ul> </li> <li>单模态与多模态的权衡</li> <li>性能与复杂度的权衡</li> <li>性能与鲁棒性的权衡</li> </ul> <h2 id="结论" class="relative group">结论 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e7%bb%93%e8%ae%ba" aria-label="Anchor">#</a></span></h2><p>一个大规模的基准，统一了以前在多模态研究中互不相干的工作，重点是易用性、可及性和可重复性。</p> <h3 id="未来拓展" class="relative group">未来拓展 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e6%9c%aa%e6%9d%a5%e6%8b%93%e5%b1%95" aria-label="Anchor">#</a></span></h3><ul> <li>其他的多模态问题</li> <li>新的评价指标</li> <li>多模态迁移学习或者协同学习</li> <li>多模态多任务学习</li> </ul> <h2 id="思考" class="relative group">思考 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e6%80%9d%e8%80%83" aria-label="Anchor">#</a></span></h2><p>MultiBench把以前多模态研究中使用的公开数据集，算法，评价指标等都统一在了一个框架下，期望标准化多模态学习过程，并且能将不同的算法模型在其他模态、任务中进行比较。大而全的框架确实能为各类多模态任务提供一个baseline，但是各专业领域内的多模态模型应该是存在一些差异的，就像我们很难期待一个医生能掌握律师干的事情，然而，人工智能的发展确实很快，比人还强大的通用人工智能应该也会实现。</p> <div class="footnotes" role="doc-endnotes"> <hr> <ol> <li id="fn:1"> <p>Liang, P. P. et al. MultiBench: Multiscale Benchmarks for Multimodal Representation Learning. (2021) doi:10.48550/arXiv.2107.07502.&#160;<a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> </ol> </div>

2023/3/4
阅读更多

ClinicalBERT: 对医学文本建模用于再入院预测

<script> document.addEventListener("DOMContentLoaded", function() { renderMathInElement(document.body, { delimiters: [ {left: '$$', right: '$$', display: true}, {left: '$', right: '$', display: false}, {left: '\\(', right: '\\)', display: false}, {left: '\\[', right: '\\]', display: true} ], throwOnError : false }); }); </script> <p>使用临床文本预训练BERT然后在再入院任务中微调</p> <h2 id="引言" class="relative group">引言 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e5%bc%95%e8%a8%80" aria-label="Anchor">#</a></span></h2><p>非结构化、高维稀疏信息例如临床文本难以在临床机器学习模型中使用。临床文本中包含什么样的临床价值？更加丰富、详细。然而重症监护室医生在有限时间内需要做出最优决策，读大量的临床文本，增加工作量。</p> <p>再入院会降低患者生活质量、增加花费。这篇文章旨在建立一个出院决策模型，根据医护人员笔记动态的赋予患者30天再入院的风险。</p> <h3 id="背景" class="relative group">背景 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e8%83%8c%e6%99%af" aria-label="Anchor">#</a></span></h3><p>临床文本会有缩写、黑话、不标准的语法结构，从临床文本中学习有用的表征具有挑战。以往的方法无法捕捉获取临床意义的文本长程依赖，介绍BERT，以及基于BERT已经开展的工作，已经有人把BERT用在临床文本了，本文在再入院任务上评估改进ClinicalBERT并且在更长的序列上进行预训练。</p> <p>介绍前人在ICU再入院预测上的工作，缺点：大多数工作都只用了出院的信息，ClinicalBERT使用患者住院整个时间段信息。</p> <h3 id="该工作的重要性" class="relative group">该工作的重要性 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e8%af%a5%e5%b7%a5%e4%bd%9c%e7%9a%84%e9%87%8d%e8%a6%81%e6%80%a7" aria-label="Anchor">#</a></span></h3><ul> <li>用出院信息来预测意味着减少了再入院风险的机会少了，都要出院了，此刻告诉有再入院的风险，难以采取措施；</li> <li>由于医院已经有很多误报警，医疗模型需要高的PPV<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup>，该模型同其他模型相比有最高的recall<sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup>；</li> <li>模型中attention能用于可视化解释。</li> </ul> <h2 id="方法" class="relative group">方法 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e6%96%b9%e6%b3%95" aria-label="Anchor">#</a></span></h2><h3 id="什么是bert" class="relative group">什么是BERT <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e4%bb%80%e4%b9%88%e6%98%afbert" aria-label="Anchor">#</a></span></h3><p>BERT是基于transformer编码器架构的深度神经网络，它用于学习文本的嵌入表达。</p> <ul> <li>自注意力机制</li> <li>BERT模型通过2个无监督任务进行预训练：掩码模型和下一个句子预测。</li> </ul> <h3 id="临床文本嵌入" class="relative group">临床文本嵌入 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e4%b8%b4%e5%ba%8a%e6%96%87%e6%9c%ac%e5%b5%8c%e5%85%a5" aria-label="Anchor">#</a></span></h3><ul> <li>先分词成token<sup id="fnref:3"><a href="#fn:3" class="footnote-ref" role="doc-noteref">3</a></sup>，这里是子词粒度的tokenization<sup id="fnref:4"><a href="#fn:4" class="footnote-ref" role="doc-noteref">4</a></sup></li> <li>ClinicalBert的token包括子词、分段嵌入、位置嵌入相加后的结果 <ul> <li>分段嵌入是当多个序列输入时，表示当前的token属于哪一段</li> <li>位置嵌入即在输入序列中token的位置</li> </ul> </li> </ul> <h3 id="自注意力机制" class="relative group">自注意力机制 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e8%87%aa%e6%b3%a8%e6%84%8f%e5%8a%9b%e6%9c%ba%e5%88%b6" aria-label="Anchor">#</a></span></h3><p>用于输入token之间的关系捕捉</p> <h3 id="预训练" class="relative group">预训练 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e9%a2%84%e8%ae%ad%e7%bb%83" aria-label="Anchor">#</a></span></h3><p>BERT是在BooksCorpus和Wikipedia中预训练的，临床文本黑话缩写，与一般文本可能语法也不一样，需要在临床文本中进行预训练。损失函数是预测掩码单词任务和预测两个句子是否连续任务损失函数之和。</p> <h3 id="微调" class="relative group">微调 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e5%be%ae%e8%b0%83" aria-label="Anchor">#</a></span></h3><p>在再入院任务中微调 $$P(readmit = 1 | h_{[cls]}) = \sigma(Wh_{[cls]})$$ 式中W为参数，h为BERT模型输出。</p> <h2 id="实验" class="relative group">实验 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e5%ae%9e%e9%aa%8c" aria-label="Anchor">#</a></span></h2><h3 id="数据" class="relative group">数据 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e6%95%b0%e6%8d%ae" aria-label="Anchor">#</a></span></h3><p>MIMIC-III中2083180份去隐私化后的文本，五折每一轮其中四折预训练，最后一折微调</p> <h3 id="实证研究i" class="relative group">实证研究I <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e5%ae%9e%e8%af%81%e7%a0%94%e7%a9%b6i" aria-label="Anchor">#</a></span></h3><ul> <li>在临床语言建模中ClinicalBERT与BERT进行比较：预测掩码token以及2个句子是否连续任务中均优于BERT</li> <li>定性分析：专家给出相似医学概念，ClinicalBERT学习嵌入表达后，进行降维可视化，发现相近</li> <li>定量分析：采用相似度度量公式计算表征之前相似度，然后与专家打分的相似度进行关联分析计算pearson相关系数</li> </ul> <h3 id="实证研究ii" class="relative group">实证研究II <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e5%ae%9e%e8%af%81%e7%a0%94%e7%a9%b6ii" aria-label="Anchor">#</a></span></h3><ul> <li>再入院队列：34560患者，2963再入院，42358负样本，<em>这里为啥有这么多负样本？</em></li> <li>调整后的再入院预测： $$P(readmit = 1|h_{patient}) = \frac{P^n_{max}+P^n_{mean}n/c}{1+n/c}$$ <ul> <li>有些文本是比较重要，有些文本对再入院预测不重要，所以要包括最大的概率</li> <li>噪声会降低性能，消除噪音的方法还是取大多数值的平均，如果序列越长，噪声出现的可能性越大，所以需要平均值的权重越大，引入了n/c作为比例因子</li> <li>分母则是用于概率归一化到0,1区间</li> </ul> </li> <li>评估指标 <ul> <li>AUROC</li> <li>AUPRC</li> <li>RP80：准确度为80%时候到召回率</li> </ul> </li> <li>模型比较：Bag of words，BI-LSTM，BERT</li> <li>用出院记录来进行再入院预测</li> <li>用24-48小时数据预测，以及48-72小时数据预测</li> <li>可解释性 <ul> <li>给出一句话的self-attention权重示意图</li> </ul> </li> </ul> <h2 id="讨论" class="relative group">讨论 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e8%ae%a8%e8%ae%ba" aria-label="Anchor">#</a></span></h2><p>建议在私有数据集上重新训练后在下游任务中使用</p> <h2 id="代码" class="relative group">代码 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e4%bb%a3%e7%a0%81" aria-label="Anchor">#</a></span></h2><ul> <li>源码：<a href="https://github.com/kexinhuang12345/clinicalBERT" target="_blank" rel="noreferrer">GitHub - kexinhuang12345/clinicalBERT: ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission (CHIL 2020 Workshop)</a></li> <li>模型：<a href="https://huggingface.co/emilyalsentzer/Bio_ClinicalBERT" target="_blank" rel="noreferrer">emilyalsentzer/Bio_ClinicalBERT · Hugging Face</a></li> </ul> <h2 id="思考" class="relative group">思考 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e6%80%9d%e8%80%83" aria-label="Anchor">#</a></span></h2><p>自chatgpt后，大型语言模型受到广泛关注，医学语言模型的发展似乎有多种路径，一种是直接在通用文本上预训练，一种是在医学文本中预训练，或是通用模型在领域微调，个人感觉应该是第三种效果会较好。</p> <div class="footnotes" role="doc-endnotes"> <hr> <ol> <li id="fn:1"> <p>Huang, K., Altosaar, J. &amp; Ranganath, R. ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission. in <em>CHIL</em> (arXiv, 2020). doi:<a href="https://doi.org/10.48550/arXiv.1904.05342" target="_blank" rel="noreferrer">10.48550/arXiv.1904.05342</a>.&#160;<a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:2"> <p>PPV: 阳性预测里面真正的阳性比例&#160;<a href="#fnref:2" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:3"> <p>recall: 正样本中实际预测为正，即真阳性率&#160;<a href="#fnref:3" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:4"> <p>token: 将原始文本切分成子单元的过程就叫做Tokenization，子单元即token&#160;<a href="#fnref:4" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> </ol> </div>

2023/3/1
阅读更多

Embedding是什么？

<script> document.addEventListener("DOMContentLoaded", function() { renderMathInElement(document.body, { delimiters: [ {left: '$$', right: '$$', display: true}, {left: '$', right: '$', display: false}, {left: '\\(', right: '\\)', display: false}, {left: '\\[', right: '\\]', display: true} ], throwOnError : false }); }); </script> <p> <figure><img src="https://cdn.jsdelivr.net/gh/jmwyf/pichosting@master/embedding.png" alt="" class="mx-auto my-0 rounded-md" /> </figure> </p> <h2 id="背景" class="relative group">背景 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e8%83%8c%e6%99%af" aria-label="Anchor">#</a></span></h2><p>在nlp领域，如何把词进行编码成数字，从而能输入到数学模型是需要考虑的：</p> <p>索引编码<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup>：</p> <ul> <li>整数编码，特征之间的关系无法捕捉</li> </ul> <p>one-hot编码的缺点：</p> <ul> <li>对于具有非常多类型的类别变量，变换后的向量维数过于巨大，且过于稀疏。</li> <li>映射之间完全独立，并不能表示出不同类别之间的关系。</li> </ul> <h2 id="embedding是什么" class="relative group">Embedding是什么 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#embedding%e6%98%af%e4%bb%80%e4%b9%88" aria-label="Anchor">#</a></span></h2><p>嵌入是将正整数（索引值）转换为固定尺寸的稠密向量<sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup>。这句话来着keras文档中对embedding层的解释，非常概括，不太容易理解，但确实概括了要干的事情。</p> <p>比如一句话，“我爱中国”对应的索引为[0,1,2,3]，要将这个索引转化为固定大小且稠密的向量来表示，而不是稀疏的one-hot编码。可以表示为$[[0.2, 0.5], [0.6,-0.1], [0.8, 0.4], [0.5, 0.5]]$，。</p> <p>词嵌入通常是8-1024维度，根据数据量的大小来调整，高维度的嵌入能更好的捕捉词之间的关系，但是需要更多的数据来训练。</p> <h2 id="embedding是如何实现的" class="relative group">Embedding是如何实现的 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#embedding%e6%98%af%e5%a6%82%e4%bd%95%e5%ae%9e%e7%8e%b0%e7%9a%84" aria-label="Anchor">#</a></span></h2><p>通过Embedding层实现，embedding层可以看作是一张从索引映射到稠密向量的查找表，当使用embedding层的时候，embedding层和神经网络其他层一样，权重是随机初始化的。根据你的训练任务，embedding层通过反向传播逐渐调整。</p> <p>embedding层的具体结构即[索引长度，emb维度]的权重矩阵也可以看作查询表，输入为整数索引，对应权重矩阵即词嵌入。skip-gram模型的前半部分即词嵌入。</p> <p>例如在tensorflow中，用于句子分类时的嵌入层，输入是整数索引，经过嵌入层、池化层、全连接输入训练可以得到嵌入层权重，即词嵌入。</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="n">embedding_dim</span><span class="o">=</span><span class="mi">16</span> </span></span><span class="line"><span class="cl"><span class="n">model</span> <span class="o">=</span> <span class="n">Sequential</span><span class="p">([</span>  </span></span><span class="line"><span class="cl"> <span class="n">vectorize_layer</span><span class="p">,</span>  </span></span><span class="line"><span class="cl"> <span class="n">Embedding</span><span class="p">(</span><span class="n">vocab_size</span><span class="p">,</span> <span class="n">embedding_dim</span><span class="p">,</span> <span class="n">name</span><span class="o">=</span><span class="s2">&#34;embedding&#34;</span><span class="p">),</span>  </span></span><span class="line"><span class="cl"> <span class="n">GlobalAveragePooling1D</span><span class="p">(),</span>  <span class="n">Dense</span><span class="p">(</span><span class="mi">16</span><span class="p">,</span> <span class="n">activation</span><span class="o">=</span><span class="s1">&#39;relu&#39;</span><span class="p">),</span>  </span></span><span class="line"><span class="cl"> <span class="n">Dense</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span> </span></span><span class="line"><span class="cl"> <span class="p">])</span> </span></span></code></pre></div><h2 id="应用" class="relative group">应用 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e5%ba%94%e7%94%a8" aria-label="Anchor">#</a></span></h2><p>最常用的就是词嵌入表达，但是万物可嵌入。Embedding在输入数据没有较好的数据表示时，能将输入数据根据下游任务转化为可学习的高维度向量表示，比如输入的为单词、图片或者输入的为空间位置等。</p> <p>mnist数据集中的图片，可以通过嵌入层来表示，如下图所示，每个点代表一个图片(10000*784)，通过嵌入层，将图片的像素点转化为稠密的向量，然后通过t-SNE/pca降维，可以看到图片的空间分布。(generated by copilot) <figure><img src="https://cdn.jsdelivr.net/gh/jmwyf/pichosting@master/mnistembedding.png" alt="" class="mx-auto my-0 rounded-md" /> </figure> </p> <p>在进行特征工程时，很难捕捉空间(时间)维度。通过使用深度学习嵌入层，我们可以通过提供一系列用户行为(作为索引)作为模型的输入来有效地捕捉这个空间维度。</p> <p>我的博客即将同步至腾讯云开发者社区，邀请大家一同入驻：https://cloud.tencent.com/developer/support-plan?invite_code=2cy4t3peazy8s</p> <div class="footnotes" role="doc-endnotes"> <hr> <ol> <li id="fn:1"> <p><a href="https://www.tensorflow.org/text/guide/word_embeddings" target="_blank" rel="noreferrer">Word embeddings  |  Text  |  TensorFlow</a>&#160;<a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:2"> <p><a href="https://keras.io/zh/layers/embeddings/" target="_blank" rel="noreferrer">嵌入层 Embedding - Keras 中文文档</a>&#160;<a href="#fnref:2" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> </ol> </div>

2023/2/21
阅读更多

Skip-gram模型（1）

<script> document.addEventListener("DOMContentLoaded", function() { renderMathInElement(document.body, { delimiters: [ {left: '$$', right: '$$', display: true}, {left: '$', right: '$', display: false}, {left: '\\(', right: '\\)', display: false}, {left: '\\[', right: '\\]', display: true} ], throwOnError : false }); }); </script> <h2 id="简介" class="relative group">简介 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e7%ae%80%e4%bb%8b" aria-label="Anchor">#</a></span></h2><p>Skip-gram<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup>属于Word2Vec的一种，给定input，预测上下文，而CBOW（见补充）是通过上下文来预测input。</p> <p>Word2Vec模型分为两个步骤<sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup>：</p> <ul> <li>建立模型，这类方法与自编码模型有点像，建模不是最终目的；</li> <li>通过模型获取嵌入词向量。</li> </ul> <h2 id="模型细节" class="relative group">模型细节 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e6%a8%a1%e5%9e%8b%e7%bb%86%e8%8a%82" aria-label="Anchor">#</a></span></h2><p>整体框架图</p> <h3 id="输入层" class="relative group">输入层 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e8%be%93%e5%85%a5%e5%b1%82" aria-label="Anchor">#</a></span></h3><p>词不能直接输入神经网络模型，比如训练样本中有10000个不同的词，将某个词“我”或者“我们”进行one-hot编码，形成10000维的向量，其中“我”的地方为1，其他均为0。对于Skip-gram输入单个词向量，输出就是这个词附近的词组成的向量。</p> <p>模型输入输出均为10000维度向量$\{0,1\}^{10000}$</p> <h3 id="隐含层" class="relative group">隐含层 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e9%9a%90%e5%90%ab%e5%b1%82" aria-label="Anchor">#</a></span></h3><p>假如想用300个特征来表征词，那么隐含层单个神经元权重为[10000, 1]，300个神经元为[10000, 300] <figure><img src="https://cdn.jsdelivr.net/gh/jmwyf/pichosting@master/sghidden.png" alt="" class="mx-auto my-0 rounded-md" /> </figure> </p> <p>上图可以理解为输入经过隐含层作用刚好得到单词的表征。输入和隐含层两部分可以进行词嵌入即embedding。</p> <h3 id="输出层" class="relative group">输出层 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e8%be%93%e5%87%ba%e5%b1%82" aria-label="Anchor">#</a></span></h3><p>模型输出为10000维度向量，要达到这样的输出层为10000个神经元，并且这些神经元输出和为1，使用softmax函数来达到这种效果。以下两个等式展示一个词向量在神经网络中的前向传播。</p> <p>$$ \begin{bmatrix} 1 &amp; 10000 \end{bmatrix} \times \begin{bmatrix} 10000 &amp; 300 \end{bmatrix} = \begin{bmatrix} 1 &amp; 300 \end{bmatrix} \ $$</p> <p>$$ \begin{bmatrix} 1 &amp; 300 \end{bmatrix} \times \begin{bmatrix} 300 &amp; 10000 \end{bmatrix} = \begin{bmatrix} 1 &amp; 10000 \end{bmatrix} \ $$</p> <h2 id="补充" class="relative group">补充 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e8%a1%a5%e5%85%85" aria-label="Anchor">#</a></span></h2><p>skip-gram名字由来<sup id="fnref:3"><a href="#fn:3" class="footnote-ref" role="doc-noteref">3</a></sup>：</p> <p>首先n-gram是一系列连续的词（tokens），而skip-gram，或者skip-n-gram，skip的是token之间的gap，jumps over the是一个3-gram，那么(jumps, the)刚好skip了一个gram (over)。</p> <p>什么是Softmax？<sup id="fnref:4"><a href="#fn:4" class="footnote-ref" role="doc-noteref">4</a></sup></p> <p>Softmax从字面上来说，可以分成soft和max两个部分。max故名思议就是最大值的意思。Softmax的核心在于soft，而soft有软的含义，与之相对的是hard硬。很多场景中需要我们找出数组所有元素中值最大的元素，实质上都是求的hardmax。hardmax最大的特点就是只选出其中一个最大的值，即非黑即白。Softmax的含义就在于不再唯一的确定某一个最大值，而是为每个输出分类的结果都赋予一个概率值，表示属于每个类别的可能性。经过使用指数形式的Softmax函数能够将差距大的数值距离拉的更大。</p> <p>CBOW是什么？</p> <p>Continuous Bag of Word Model连续词带模型，通过上下文来预测中间的词</p> <div class="footnotes" role="doc-endnotes"> <hr> <ol> <li id="fn:1"> <p><a href="http://mccormickml.com/2016/04/19/word2vec-tutorial-the-skip-gram-model/" target="_blank" rel="noreferrer">Word2Vec Tutorial - The Skip-Gram Model · Chris McCormick</a>&#160;<a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:2"> <p><a href="https://zhuanlan.zhihu.com/p/27234078?from=singlemessage" target="_blank" rel="noreferrer">理解 Word2Vec 之 Skip-Gram 模型 - 知乎</a>&#160;<a href="#fnref:2" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:3"> <p><a href="https://www.zhihu.com/question/302594410?utm_id=0" target="_blank" rel="noreferrer">skip-gramm模型skip了什么？为什么叫skip-gramm模型？ - 知乎</a>&#160;<a href="#fnref:3" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:4"> <p><a href="https://zhuanlan.zhihu.com/p/105722023" target="_blank" rel="noreferrer">一文详解Softmax函数 - 知乎</a>&#160;<a href="#fnref:4" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> </ol> </div>

2023/2/5
阅读更多

公开重症监护数据库MIMIC代码仓库介绍

<div class="lead !mb-9 text-xl"> 《The MIMIC Code Repository: Enabling reproducibility in critical care research》论文 </div> <h2 id="引言" class="relative group">引言 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e5%bc%95%e8%a8%80" aria-label="Anchor">#</a></span></h2><ul> <li>科学结果的可重复性越来越受到关注<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup>；</li> <li>医疗领域进入数字化革命（本文是2017年接收），引出形成MIMIC-III数据库；</li> <li>EHR二次分析需要临床专家和数据科学家的合作，在EHR数据库上推导或者定义一些概念是需要资源的，对于没有特别强的临床背景或者数据科学技能的人来说巨大障碍；</li> <li>该文介绍MIMIC代码仓库，介绍与重症相关概念的导出以及相关假设条件等；</li> <li>公开数据已经逐渐有了，公开相应的数据代码同样重要。加速并提升未来研究的一致性以及有效性。</li> </ul> <h2 id="代码仓库详情" class="relative group">代码仓库详情 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e4%bb%a3%e7%a0%81%e4%bb%93%e5%ba%93%e8%af%a6%e6%83%85" aria-label="Anchor">#</a></span></h2><ul> <li>Concepts <ul> <li>从电子病历中提取重要概念的代码。比如提取AKI的模块</li> </ul> </li> <li>Executable documents <ul> <li>可执行的Notebooks文件，可重复的示例研究或者教程</li> </ul> </li> <li>Community <ul> <li>建立公开讨论便于社区成员贡献</li> </ul> </li> </ul> <h3 id="概念concepts" class="relative group">概念concepts <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e6%a6%82%e5%bf%b5concepts" aria-label="Anchor">#</a></span></h3><p>代码库中常用的概念</p> <h4 id="疾病严重程度评分severity-of-illness-scores" class="relative group">疾病严重程度评分Severity of illness scores <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e7%96%be%e7%97%85%e4%b8%a5%e9%87%8d%e7%a8%8b%e5%ba%a6%e8%af%84%e5%88%86severity-of-illness-scores" aria-label="Anchor">#</a></span></h4><p>在回顾性数据库中难以计算</p> <ul> <li>大多都是在前瞻性实验中获取的；</li> <li>常规收集的数据缺相应元素。有些特征未纳入结构化电子病历系统，另外则是对某种情况的患者没有统一的协议来定义状态</li> </ul> <p>目前MIMIC代码库中有：</p> <ul> <li>acute physiology score(APS)-III</li> <li>simplified acute physiology score(SAPS)</li> <li>SAPS-II</li> <li>Oxford acute severity of illness score(OASIS)</li> </ul> <h4 id="器官衰竭organ-dysfunction-scores" class="relative group">器官衰竭Organ dysfunction scores <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e5%99%a8%e5%ae%98%e8%a1%b0%e7%ab%adorgan-dysfunction-scores" aria-label="Anchor">#</a></span></h4><p>SOFA计算方式不同，由于GCS评分定义不同</p> <p>Sequential Organ Failure Assessment(SOFA), Logistic Organ Dysfunction system(LODS)</p> <h4 id="治疗时间time-of-treatment" class="relative group">治疗时间Time of treatment <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e6%b2%bb%e7%96%97%e6%97%b6%e9%97%b4time-of-treatment" aria-label="Anchor">#</a></span></h4><p>由于数据获取的限制，许多药物和确切的治疗时间无法得出，需要根据临床经验识别其他可替代的数据</p> <ul> <li>机械通气时长：识别机械通气时长需要复杂的逻辑规则（文中图3）</li> <li>血管加压药物使用</li> <li>CRRT</li> </ul> <h4 id="脓毒症sepsis" class="relative group">脓毒症sepsis <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e8%84%93%e6%af%92%e7%97%87sepsis" aria-label="Anchor">#</a></span></h4><p>sepsis定义有多种版本，这里给出了<em>Angus 2001，Martin 2003，Iwashyna 2014</em>三个版本</p> <h4 id="共病comorbidities" class="relative group">共病Comorbidities <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e5%85%b1%e7%97%85comorbidities" aria-label="Anchor">#</a></span></h4><p>给出了4个版本</p> <ul> <li><em>Elixhauser A 1998</em></li> <li>American Health and Research Quality group（AHRQ）</li> <li><em>Quan 2005</em></li> <li><em>Van Walraven 2009</em></li> </ul> <h4 id="concept指南" class="relative group">concept指南 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#concept%e6%8c%87%e5%8d%97" aria-label="Anchor">#</a></span></h4><p> <figure><img src="https://cdn.jsdelivr.net/gh/jmwyf/pichosting@master/mimicroadmap.png" alt="" class="mx-auto my-0 rounded-md" /> </figure> </p> <h3 id="可执行文档" class="relative group">可执行文档 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e5%8f%af%e6%89%a7%e8%a1%8c%e6%96%87%e6%a1%a3" aria-label="Anchor">#</a></span></h3><p>当数据和代码都公开可获取，提供一种研究可以被重现的框架，基于Rmd或notebook给出实例。</p> <ul> <li><em>Hsu 2015</em>研究复现 <ul> <li>indwelling arterial catheters and their association with in-hospital mortality for hemodynamically stable patients with respiratory failure</li> <li><code>aline.ipynb</code>提取数据</li> <li><code>aline.Rmd</code>数据分析</li> </ul> </li> <li>教程 <ul> <li>definition of CRRT</li> <li>introduction to SQL</li> <li>a step-by-step guide to selecting a study cohort</li> <li>an outline of the data-capture process</li> </ul> </li> </ul> <h3 id="社区" class="relative group">社区 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e7%a4%be%e5%8c%ba" aria-label="Anchor">#</a></span></h3><p>让研究人员和数据维护人员、临床人员共同提升代码</p> <h2 id="结论" class="relative group">结论 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e7%bb%93%e8%ae%ba" aria-label="Anchor">#</a></span></h2><p>公开数据库的案例已经不少，为了让研究更加透明，也需要公开相应数据分析和数据处理的代码</p> <h2 id="补充" class="relative group">补充 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e8%a1%a5%e5%85%85" aria-label="Anchor">#</a></span></h2><ul> <li>代码库地址：https://github.com/MIT-LCP/mimic-code <ul> <li>之前以MIMIC-III为主，现在mimic-iii和mimic-iv合并在一起了</li> </ul> </li> <li>mimic数据库为了让研究者访问更加方便，很大一个改变是部署在云上比如google的云平台，云平台上需要big query语法来访问，所以现在代码库关于数据提取的代码更新以big query为主，需要通过脚本转化为适合postgres语法 <ul> <li>Open a terminal in the <code>concepts</code> folder.</li> <li>Run <a href="https://github.com/MIT-LCP/mimic-code/blob/main/mimic-iv/concepts/convert_bigquery_to_postgres.sh" target="_blank" rel="noreferrer">convert_bigquery_to_postgres.sh</a>. <ul> <li>e.g. <code>bash convert_bigquery_to_postgres.sh</code></li> <li>This file outputs the scripts to the <a href="https://github.com/MIT-LCP/mimic-code/blob/main/mimic-iv/concepts/postgres" target="_blank" rel="noreferrer">postgres</a> subfolder after applying a few changes.</li> <li>This also creates the <code>postgres_make_concepts.sql</code> script in the postgres subfolder.</li> </ul> </li> </ul> </li> <li>从代码仓库导出的概念concepts都放到<code>mimic_derived</code>数据集里</li> </ul> <div class="footnotes" role="doc-endnotes"> <hr> <ol> <li id="fn:1"> <p>Johnson, A. E. W., Stone, D. J., Celi, L. A. &amp; Pollard, T. J. The MIMIC Code Repository: enabling reproducibility in critical care research. <em>J Am Med Inform Assn</em> <strong>25</strong>, 32–39 (2018).&#160;<a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> </ol> </div>

2023/2/4
阅读更多

公开重症监护数据库MIMIC-IV介绍

<h2 id="背景" class="relative group">背景 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e8%83%8c%e6%99%af" aria-label="Anchor">#</a></span></h2><ul> <li>本文<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup>首先介绍mimic项目的由来，医院数据归档系统都不是为研究设计的，难以访问以及查询；</li> <li>MIMIC从0-4发展简介；</li> <li>其他数据库简介 <ul> <li>eICU-CRD，美国多中心200859例ICU记录</li> <li>AmsterdamUMCdb，荷兰单医学中心20109例患者23106次入院记录</li> <li>HiRID<a href="https://hirid.intensivecare.ai/data-details" target="_blank" rel="noreferrer">Data details</a>，瑞士34000入院患者高分辨率数据（每两分钟一个数据点，712中常规手机生理数据） <ul> <li>HiRID has a higher time resolution than other published datasets, most importantly for bedside monitoring with most parameters recorded every 2 minutes</li> <li>去隐私化方法可以借鉴</li> </ul> </li> <li>PIC，中国12881例患者13941次儿科ICU记录</li> </ul> </li> <li>问题，当前数据库都只是单模态的数据或各有特点，且临床实践发展变化快，数据库需要不断更新完善；</li> <li>MIMIC-IV是当代的综合多模态数据库</li> </ul> <h2 id="方法" class="relative group">方法 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e6%96%b9%e6%b3%95" aria-label="Anchor">#</a></span></h2><p>mimic-iv数据库是怎么建的？</p> <ul> <li>获取、转化、去隐私化</li> </ul> <h3 id="获取" class="relative group">获取 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e8%8e%b7%e5%8f%96" aria-label="Anchor">#</a></span></h3><p>BIDMC医院常规的临床数据存储在microsoft SQL中，通过VPN转移到MIT服务器的PostgreSQL，补充数据如ICD、患者死亡信息等外部导入到PostgreSQL</p> <ul> <li>队列：2008-2019，18岁以上</li> <li>床旁信息来自于MetaVision，医院EHR会通过HL7推信息到MetaVision</li> <li>外部数据主要说明DRGs和ICD是如何处理以及死亡信息如何处理 <ul> <li>DRGs和ICD由于数据库跨度时间长以及不同版本，在数据库中把这些都导入了</li> <li>死亡信息通过与马萨诸塞州生命记录和统计登记处进行匹配，而非社保档案</li> </ul> </li> </ul> <h3 id="转化" class="relative group">转化 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e8%bd%ac%e5%8c%96" aria-label="Anchor">#</a></span></h3><p>转化有两条原则</p> <ul> <li>与MIMIC-III保持兼容</li> <li>尽量减少处理过程让公开数据与临床实践数据保持一致</li> </ul> <p>数据被分为三个组：hosp、icu和note</p> <ul> <li>hosp：admission/discharge/transfer（ADT），实验室检查结果，微生物培养，处方，管理数据</li> <li>icu：患者出入量、输液、操作、记录到的观测值等</li> <li>note：出院总结和放射学报告，也创建了相对应的自由文本结构化表“实体-属性-值”</li> </ul> <h3 id="去隐私化" class="relative group">去隐私化 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e5%8e%bb%e9%9a%90%e7%a7%81%e5%8c%96" aria-label="Anchor">#</a></span></h3><ul> <li>遵从The Health Insurance Portability and Accountability Act（HIPAA）条款规定了18项标识符，包括姓名、地址、年龄等需要去掉</li> <li>日期移动，但时间点间距保留</li> <li>结合了两个公开的算法<sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup><sup id="fnref:3"><a href="#fn:3" class="footnote-ref" role="doc-noteref">3</a></sup>从自由文本中移除个人健康信息（PHI） <ul> <li>两个算法都没捕捉到的，从数据库中移除？这个没写具体怎么做</li> </ul> </li> </ul> <h2 id="数据记录结果" class="relative group">数据记录结果 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e6%95%b0%e6%8d%ae%e8%ae%b0%e5%bd%95%e7%bb%93%e6%9e%9c" aria-label="Anchor">#</a></span></h2><p>这一部分类似传统论文的结果，详细介绍了hosp、icu、note模块里面的数据情况，对各表进行介绍</p> <h3 id="hosp" class="relative group">Hosp <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#hosp" aria-label="Anchor">#</a></span></h3><p>首先介绍模块里主要键值以及表之间连接关系，与MIMIC-III一致</p> <p>患者基本信息：patients、admissions和transfers表</p> <ul> <li>患者时间信息，anchor_year、anchor_age、anchor_year_group这几个重要项 <ul> <li>anchor_year锚定年份，由于时间都是去隐私化的平移过的，这里可以看作平移之后的参考年份</li> <li>anchor_year_group是真实年份的区间</li> </ul> </li> <li>患者死亡信息最多到患者出院1年为止</li> </ul> <p>管理信息：services、poe、poe_detail表</p> <ul> <li>services 患者住院期间所受到的医疗服务 <ul> <li> <figure><img src="https://cdn.jsdelivr.net/gh/jmwyf/pichosting@master/services.png" alt="" class="mx-auto my-0 rounded-md" /> </figure> </li> </ul> </li> <li>poe 医嘱录入系统：治疗和操作</li> </ul> <p>计费信息：diagnoses_icd、procedures_icd、drgcodes、hcpcsevents</p> <p>检查结果：microbiologyevents、labevents</p> <p>药物信息：prescriptions、pharmacy、emar、emar_detail</p> <ul> <li>处方、药房信息；</li> <li>2016年部署electronic Medicine Administration Record，eMAR；</li> <li>看起来关系比较复杂使用得结合实例</li> </ul> <h3 id="icu" class="relative group">ICU <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#icu" aria-label="Anchor">#</a></span></h3><p>chartevents, d_items, datetimeevents, icustays, inputevents, outputevents, and procedureevents表</p> <h3 id="note" class="relative group">Note <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#note" aria-label="Anchor">#</a></span></h3><p>出院总结：</p> <ul> <li>主诉、现病史、既往病史、简要病程、体格检查和出院诊断</li> </ul> <p>放射学报告：</p> <ul> <li>x射线、CT、MRI、超声</li> </ul> <h2 id="技术验证结果" class="relative group">技术验证结果 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e6%8a%80%e6%9c%af%e9%aa%8c%e8%af%81%e7%bb%93%e6%9e%9c" aria-label="Anchor">#</a></span></h2><ul> <li>完整性检查 <ul> <li>数据库自身是完备的，不会出现某个患者的信息只出现一张表其他表都找不到的情况</li> </ul> </li> <li>一致性检查 <ul> <li>数据结果与一些常识一致</li> </ul> </li> <li>去隐私化检查</li> </ul> <h2 id="使用笔记" class="relative group">使用笔记 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e4%bd%bf%e7%94%a8%e7%ac%94%e8%ae%b0" aria-label="Anchor">#</a></span></h2><ul> <li>如何获取 <ul> <li>参加课程</li> <li>签署data use agreement, DUA</li> </ul> </li> <li>代码库以及模版</li> </ul> <h2 id="思考" class="relative group">思考 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e6%80%9d%e8%80%83" aria-label="Anchor">#</a></span></h2><ul> <li>MIMIC数据库一步一步发展已经来到了第四版，很好的体现了科学研究的可持续发展。</li> <li>与MIMIC-III论文写法不同，mimic-iv对数据库构建过程写的更加详细，可操作性更高，而mimic-iii由于没有很好的把这个过程结构化抽象分成几步显得构建细节不足，而mimic-iii论文表格比较丰富，对数据库进行了一些粗粒度的介绍，mimic-iv论文没有。个人认为iv这篇论文写的更好一点。</li> <li>伴随着数据类型越来越多、数据库越来越多，数据分析人员也需要掌握更多类型数据预处理方法。面对文本、图像、波形的多模态数据分析或者在不完备数据情况下模型的不确定性成为两个相对应的研究方向。</li> </ul> <div class="footnotes" role="doc-endnotes"> <hr> <ol> <li id="fn:1"> <p>Johnson, A. E. W. <em>et al.</em> MIMIC-IV, a freely accessible electronic health record dataset. <em>Sci Data</em> <strong>10</strong>, 1 (2023).&#160;<a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:2"> <p>Neamatullah, I. et al. Automated de-identification of free-text medical records. BMC medical informatics and decision making 8,1–17 (2008).&#160;<a href="#fnref:2" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:3"> <p>Johnson, A. E. W., Bulgarelli, L. &amp; Pollard, T. J. Deidentification of free-text medical records using pre-trained bidirectional transformers. In Proceedings of the ACM Conference on Health, Inference, and Learning, 214–221 (2020).&#160;<a href="#fnref:3" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> </ol> </div>

2023/1/17
阅读更多

MIT 6.S91 Introduction Deep Learning Notes

<script> document.addEventListener("DOMContentLoaded", function() { renderMathInElement(document.body, { delimiters: [ {left: '$$', right: '$$', display: true}, {left: '$', right: '$', display: false}, {left: '\\(', right: '\\)', display: false}, {left: '\\[', right: '\\]', display: true} ], throwOnError : false }); }); </script> <h2 id="1introduction-to-deep-learning" class="relative group">1.Introduction to Deep learning <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#1introduction-to-deep-learning" aria-label="Anchor">#</a></span></h2><ul> <li>震撼，第一节课直接放大招，用自己拍摄的视频和奥巴马合成来介绍这门课程。</li> <li><strong>不管老师在课程上讲什么，希望你们能真正的思考为什么这一步是重要而且必须的，正是这些思考才能做出真正令人惊讶的突破。</strong></li> </ul> <h2 id="2deep-sequence-model" class="relative group">2.Deep Sequence Model <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#2deep-sequence-model" aria-label="Anchor">#</a></span></h2><p>Three way to solve gradient vanish</p> <ul> <li>Gated Cells <ul> <li>LSTM <ul> <li>Forget</li> <li>Store</li> <li>Update</li> <li>Output</li> </ul> </li> </ul> </li> <li>Attention [[Transformer]]</li> </ul> <h2 id="3deep-computer-vision" class="relative group">3.Deep Computer Vision <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#3deep-computer-vision" aria-label="Anchor">#</a></span></h2><ul> <li>介绍卷积操作，是一种提取特征的方法生成feature maps（还有其他的方法可以用吗？然后效果还不错）； <ul> <li>与全连接相比的优点；</li> </ul> </li> <li><input disabled="" type="checkbox"> Fast RCNN用于目标检测，怎么实现推荐特定区域图像？</li> </ul> <ul> <li>医学图片分割</li> </ul> <ul> <li>总结： <ul> <li>原理</li> <li>CNN架构</li> <li>应用</li> </ul> </li> </ul> <h2 id="4deep-generative-models" class="relative group">4.Deep Generative Models <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#4deep-generative-models" aria-label="Anchor">#</a></span></h2><ul> <li>what 目标: 来自于一些分布中的训练样本，通过这些样本学习模型来表征这个分布；</li> <li>how 密度估计；神经网络适合来进行高维度表征；</li> <li>why <ul> <li><strong>Debiasing</strong>: Capable of uncovering underlying features in a dataset</li> <li>Outlier detection: how can we detect when we encounter something new or rare?</li> </ul> </li> </ul> <ul> <li>Latent variable representation: <ul> <li>举例事物的投影，只能看见影子即表象，而被灯光照射的实物是看不见的即隐变量；要做的是通过观察到的投影来对实物进行建模</li> </ul> </li> <li>Autoencoder: reconstruction loss <ul> <li>完全是确定性性</li> </ul> </li> </ul> <ul> <li>VAEs：normal prior + regularization <ul> <li>reconstruction loss + regularization term</li> <li>encoder: $q_\phi(z|x)$</li> <li>decoder: $p_\theta(x|z)$</li> <li>KL-divergence: $D(q_\phi(z|x)||p(z))$</li> </ul> </li> </ul> <p> <figure><img src="https://cdn.jsdelivr.net/gh/jmwyf/pichosting@master/VAEsummary.png" alt="" class="mx-auto my-0 rounded-md" /> </figure> </p> <ul> <li>GANs <ul> <li>make a generative model by having two neural networks compete with each other</li> <li>⭐️CycleGAN: domain transformations 视频开头的视频就是用这个合成</li> </ul> </li> </ul> <h2 id="5deep-reinforcement-learning" class="relative group">5.Deep reinforcement learning <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#5deep-reinforcement-learning" aria-label="Anchor">#</a></span></h2><ul> <li>Reward: $$R_t = r_t + \gamma r_{t+1} + \gamma^2 r_{t+2} + &hellip;$$</li> <li>Q-function: expected total future reward $$Q(s_t, a_t) = E[R_t|s_t, a_t]$$</li> <li>Policy: to infer the best action to take at its state, choose an action that maximizes future reward $$\pi^*(s)=\mathop{\arg\max}\limits_{s}Q(s, a)$$</li> <li>Value Learning <ul> <li>find $Q(s, a)$</li> <li>$a = \mathop{\arg\max}\limits_{a}Q(s, a)$</li> </ul> </li> <li>Police Learning <ul> <li>find $\pi(s)$</li> <li>sample $a\sim\pi(s)$</li> </ul> </li> <li><input disabled="" type="checkbox"> Deep Q Network(DQN)</li> <li><input disabled="" type="checkbox"> Policy Gradient</li> <li><input disabled="" type="checkbox"> AlphaGo</li> </ul> <h2 id="6dl-limitations-and-new-frontiers" class="relative group">6.DL Limitations and New Frontiers <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#6dl-limitations-and-new-frontiers" aria-label="Anchor">#</a></span></h2><ul> <li>limitations <ul> <li>Generalization <ul> <li>data is important</li> </ul> </li> <li>Uncertainty in Deep learning</li> <li>adversarial attack</li> <li>Algorithmic Bias</li> </ul> </li> <li>Frontiers <ul> <li>encoder <ul> <li>many real world data cannot be captured by standard encodings</li> <li><input disabled="" type="checkbox"> GCN（Graph Convolutional Networks）</li> </ul> </li> <li>Automated AI</li> </ul> </li> </ul> <h2 id="7-lidar-for-autonomous-driving" class="relative group">7. LiDAR for Autonomous Driving <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#7-lidar-for-autonomous-driving" aria-label="Anchor">#</a></span></h2><p>@INNOVIZ</p> <ul> <li>Camera Vs LiDAR <ul> <li>互补，视线不好的情况</li> <li>冗余能保证准确</li> </ul> </li> <li>Safety and Comfort</li> </ul> <h2 id="8-automatic-speech-recognition" class="relative group">8. Automatic Speech Recognition <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#8-automatic-speech-recognition" aria-label="Anchor">#</a></span></h2><p>@Rev</p> <ul> <li><input disabled="" type="checkbox"> Conformer</li> <li><input disabled="" type="checkbox"> CTC</li> </ul> <h2 id="9-ai-fore-science" class="relative group">9. AI fore Science <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#9-ai-fore-science" aria-label="Anchor">#</a></span></h2><p>Principled AI Algorithms for challenging domains @Caltech</p> <h2 id="10-uncertainty-in-deep-learning" class="relative group">10. Uncertainty in Deep Learning <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#10-uncertainty-in-deep-learning" aria-label="Anchor">#</a></span></h2><p>longer version：NeurIPS 2020 Tutorial @Google AI Brain Team</p> <ul> <li>Return a distribution over predictions rather than a single prediction</li> <li>Out-of-Distribution Robustness <ul> <li>covariate shift: distribution of features changes</li> <li>open-set recognition: new classes may appear at test time</li> <li>label shift: distribution of label changes</li> </ul> </li> <li>sources of uncertainty <ul> <li>Model uncertainty <ul> <li>认知上的不确定性</li> </ul> </li> <li>Data uncertainty <ul> <li>human disagreement label noise</li> <li>measurement noise</li> <li>missing data</li> </ul> </li> </ul> </li> <li>how to compute <ul> <li><input disabled="" type="checkbox"> BDN</li> <li><input disabled="" type="checkbox"> GP</li> <li><input disabled="" type="checkbox"> Deep Ensemble</li> <li><input disabled="" type="checkbox"> MCMC</li> </ul> </li> <li>multi-input and multi output（MIMO）</li> <li>how to communicate with uncertainty?</li> </ul> <p>7-10讲很一般，一个复杂的主题，需要将背景讲清楚，公司讲东西也没啥具体细节。</p> <h2 id="ref" class="relative group">Ref <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#ref" aria-label="Anchor">#</a></span></h2><ol> <li><a href="https://www.bilibili.com/video/BV1jo4y1d7R6/?spm_id_from=333.1007.top_right_bar_window_custom_collection.content.click&amp;vd_source=c2e29329f33c2e7eb04916d212234ad6" target="_blank" rel="noreferrer">【双语字幕】MIT《深度学习导论(6.S191)》课程(2021)_哔哩哔哩_bilibili</a></li> <li><a href="http://introtodeeplearning.com/" target="_blank" rel="noreferrer">introtodeeplearning.com</a></li> <li><a href="https://www.youtube.com/watch?v=QcLlc9lj2hk&amp;list=PLtBw6njQRU-rwp5__7C0oIVt26ZgjG9NI&amp;index=4" target="_blank" rel="noreferrer">MIT 6.S191: Deep Generative Modeling - YouTube</a></li> </ol>

2023/1/13
阅读更多

《A Unified Approach to interpreting Model Predictions》论文解读

<script> document.addEventListener("DOMContentLoaded", function() { renderMathInElement(document.body, { delimiters: [ {left: '$$', right: '$$', display: true}, {left: '$', right: '$', display: false}, {left: '\\(', right: '\\)', display: false}, {left: '\\[', right: '\\]', display: true} ], throwOnError : false }); }); </script> <h2 id="introduction" class="relative group">Introduction <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#introduction" aria-label="Anchor">#</a></span></h2><ul> <li>大数据让复杂模型的优势明显</li> <li>提出一种新颖统一的方法用于模型解释 <ul> <li>用模型的方法来解释复杂模型（用魔法打败魔法）</li> <li>提出SHAP值作为各种方法近似统一特征重要度度量</li> <li>提出新的SHAP值估计方法</li> </ul> </li> </ul> <h2 id="interpretation-model-properties" class="relative group">Interpretation model properties <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#interpretation-model-properties" aria-label="Anchor">#</a></span></h2><p>描述解释模型需要有的三个性质，而现在解释方法的缺陷有哪些<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup></p> <ul> <li>局部准确性：如果x‘是x的简化特征，对应解释模型g(x&rsquo;) = f(x)，即解释模型在给定的特征情况下能解释为什么模型预测值是这么多。</li> <li>缺失性：当x&rsquo;=0的时候，贡献度$\phi$为0</li> <li>一致性：模型改变导致特征变的更重要时，贡献度也应该变大</li> </ul> <h2 id="additive-feature-attribution-methods" class="relative group">Additive Feature Attribution methods <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#additive-feature-attribution-methods" aria-label="Anchor">#</a></span></h2><p>一大类方法中解释模型是一系列二元变量的线性函数 称为<strong>Additive Feature Attribution methods</strong>（AFA）相加特征归因方法 $$g(z&rsquo;) = \phi_0 + \sum_{i=1}^{M}\phi_i z&rsquo;_{i}$$ $z&rsquo; \in {0, 1}^M$</p> <h2 id="classic-shapley-value-estimation" class="relative group">Classic Shapley Value Estimation <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#classic-shapley-value-estimation" aria-label="Anchor">#</a></span></h2><p>$$\phi_{i} = \sum_{S \subseteq F\backslash{i}}\frac{|S|!(|F|-|S|-1)!}{|F|!}[f_{S\cup{i}}(x_{S\cup{i}})-f_{S(x_S)}]$$ 基于上面公式精确计算很麻烦，特征的排列有$2^F$种，计算量巨大</p> <h2 id="shap" class="relative group">SHAP <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#shap" aria-label="Anchor">#</a></span></h2><p>SHAP分为模型无关和模型相关两类方法用来近似求解，模型无关的代表是kernelSHAP，而模型相关的代表则有DeepSHAP和TreeSHAP，一个是针对深度学习，一个是针对树模型。</p> <h2 id="kernel-shap" class="relative group">Kernel SHAP <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#kernel-shap" aria-label="Anchor">#</a></span></h2><h3 id="lime" class="relative group">LIME <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#lime" aria-label="Anchor">#</a></span></h3><p>LIME<sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup>（Local interpretable model-agnostic explanations)（why should I trust you: Explaining the predictions of any classifier）通过生成的包含需要解释点周围的扰动数据和基于黑箱模型预测结果的数据集，训练一个可以解释的模型，比如逻辑回归、决策树，这个可解释模型需要在解释点周围达到较好的效果。 $$\xi = \mathop{\arg\min}\limits_{g\in G}L(f,g,\pi_x) + \Omega(g)$$</p> <ul> <li>f为需解释模型</li> <li>g为可能的解释模型</li> <li>$\pi_x$为定义实例周围多大范围</li> </ul> <p>算法过程：</p> <ul> <li>选择需要解释感兴趣的实例</li> <li>对其进行扰动，并得到黑箱模型对应结果产生新数据集</li> <li>根据与实例的接近程度，对新数据集进行赋予权重</li> <li>基于新数据集和上述损失函数求解可解释模型</li> <li>解释预测值 <figure><img src="https://cdn.jsdelivr.net/gh/jmwyf/pichosting@master/weightshap.png" alt="" class="mx-auto my-0 rounded-md" /> </figure> Figure 3: Toy example to present intuition for LIME</li> </ul> <h3 id="kernelshaplinear-lime--shapley-value" class="relative group">KernelSHAP（Linear LIME + Shapley value） <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#kernelshaplinear-lime--shapley-value" aria-label="Anchor">#</a></span></h3><p>LIME（Local interpretable model-agnostic explanations）方法的拓展，通过修改LIME需要求解loss等式参数，其主要思路是利用核变换，让$\phi$符合shapley value<sup id="fnref:3"><a href="#fn:3" class="footnote-ref" role="doc-noteref">3</a></sup> 算法过程<sup id="fnref:4"><a href="#fn:4" class="footnote-ref" role="doc-noteref">4</a></sup>：</p> <ul> <li>根据$z_k^{&rsquo;} \in {0, 1}^M$选择k个样本</li> <li>将$z_k^{&rsquo;} \in {0, 1}^M$转化为原始特征值并计算黑箱模型预测值</li> <li>基于SHAP kernel计算$z_k^{&rsquo;}$ 样本权重，$z_k$里面1的个数不一样权重就不一样</li> <li>拟合线性模型</li> <li>从线性模型中返回Shapley values SHAP核为(推导过程见2补充材料)： $$\pi_{x&rsquo;}(z&rsquo;)= \frac{(M-1)}{(M choose |z&rsquo;|)|z&rsquo;|(M-|z&rsquo;|)}$$ $$L(f,g, \pi_{x&rsquo;})=\sum_{z&rsquo;\in Z}[f(h_x^{-1}(z&rsquo;))-g(z&rsquo;)]^2\pi_{x&rsquo;}(z&rsquo;)$$ x&rsquo;为简化输入，$x=h_x(x&rsquo;)$, $z&rsquo; \subseteq x&rsquo;$ 其中第二步：The function h maps 1’s to the corresponding value from the instance x that we want to explain. For tabular data, it maps 0’s to the values of another instance that we sample from the data. This means that we equate “feature value is absent” with “feature value is replaced by random feature value from data”.</li> </ul> <h2 id="deep-shap" class="relative group">Deep SHAP <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#deep-shap" aria-label="Anchor">#</a></span></h2><h3 id="deepliftlearning-important-features" class="relative group">DeepLIFT（Learning Important FeaTures） <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#deepliftlearning-important-features" aria-label="Anchor">#</a></span></h3><p>DeepLIFT<sup id="fnref:5"><a href="#fn:5" class="footnote-ref" role="doc-noteref">5</a></sup>方法使用神经元的激活与其“参考”进行比较，其中参考是神经元在网络获得“参考输入”时具有的激活状态（参考输入根据具体任务的内容定义）。该方法赋予每个特征重要度分数之和等于预测值与基于参考输入的预测值之间的差异<sup id="fnref:6"><a href="#fn:6" class="footnote-ref" role="doc-noteref">6</a></sup>。 能解决基于梯度方法的不足，例如参考的差异不是0的情况下梯度仍然可能是0。 $$\sum_{i=1}^n C_{\Delta x_i \Delta t} = \Delta t \tag1$$ $\Delta t = t - t^0$, 神经元输出与参考输出的差异，$\Delta x$输入相对参考输入的变化， C即特征的贡献</p> <p>$$m_{\Delta x\Delta t} = \frac{C_{\Delta x\Delta t}}{\Delta x} \tag2$$ 定义乘子multiplier，满足链式法则 算法步骤<sup id="fnref:7"><a href="#fn:7" class="footnote-ref" role="doc-noteref">7</a></sup>：</p> <ul> <li>定义参考值 <ul> <li>choosing a good reference would rely on domain-specific knowledge, and in some cases it may be best to compute DeepLIFT scores against multiple different references</li> <li>比如图像用全0或者模糊版本，基因数据用本底期望频率</li> </ul> </li> <li>区分正负贡献（2019）</li> <li>贡献度规则 <ul> <li>线性层直接是系数$w* \Delta x_i$</li> <li>非线性变化是$m_{\Delta x\Delta y} = \frac{C_{\Delta x\Delta y}}{\Delta x} =\frac{\Delta y}{\Delta x}$</li> </ul> </li> </ul> <h3 id="deepshapdeeplift--shapley-value" class="relative group">DeepSHAP(DeepLIFT + Shapley value) <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#deepshapdeeplift--shapley-value" aria-label="Anchor">#</a></span></h3><p>尽管kernelSHAP是适用于所有模型的包括深度学习模型的一种可解释方法，但是有没有能利用神经网络特性的可解释方法从而提高计算效率。 DeepLIFT计算的分数近似于Shapley value?并不是这个思路</p> <ul> <li>the Shapely values measure the average marginal effect of including an input over all possible orderings in which inputs can be included. If we define “including” an input as setting it to its actual value instead of its reference value, DeepLIFT can be thought of as a fast approximation of the Shapely values<sup id="fnref:8"><a href="#fn:8" class="footnote-ref" role="doc-noteref">8</a></sup></li> <li>Though a variety of methods exist for estimating SHAP values, we implemented a modified version of the DeepLIFT algorithm, which computes SHAP by estimating differences in model activations during backpropagation relative to a standard reference. <figure><img src="https://cdn.jsdelivr.net/gh/jmwyf/pichosting@master/components.png" alt="" class="mx-auto my-0 rounded-md" /> </figure> <figure><img src="https://cdn.jsdelivr.net/gh/jmwyf/pichosting@master/shap.png" alt="" class="mx-auto my-0 rounded-md" /> </figure> figure from ref[3]</li> <li>基于Shapley value的定义以及公式可以看出重要的一部分即边际效应，即模型包含该特征减去未包含该部分。在上述一个神经网络模块里面，特征顺序选择都不存在。在认为包含特征即相对于参考输入是真实输入的情况下，把包含特征后乘子直接链式法则做为SHAP值近似公式</li> <li>在上述简单网络组件里面，输入到输出之间可以看作线性近似从而得到公式16</li> <li>把用实际值代替参考值看作是包含某个特征，DeepLIFT方法与DeepSHAP似乎看不到区别?</li> </ul> <div class="footnotes" role="doc-endnotes"> <hr> <ol> <li id="fn:1"> <p><a href="https://blog.ml.cmu.edu/2020/08/31/6-interpretability/" target="_blank" rel="noreferrer">6 – Interpretability – Machine Learning Blog | ML@CMU | Carnegie Mellon University</a>&#160;<a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:2"> <p>Ribeiro, M. T., Singh, S. &amp; Guestrin, C. ‘Why Should I Trust You?’: Explaining the Predictions of Any Classifier. <em>Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining</em> 1135–1144 (2016) doi:<a href="https://doi.org/10.1145/2939672.2939778" target="_blank" rel="noreferrer">10.1145/2939672.2939778</a>.&#160;<a href="#fnref:2" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:3"> <p><a href="https://papers.nips.cc/paper/2017/hash/8a20a8621978632d76c43dfd28b67767-Abstract.html" target="_blank" rel="noreferrer">A Unified Approach to Interpreting Model Predictions</a>&#160;<a href="#fnref:3" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:4"> <p><a href="https://christophm.github.io/interpretable-ml-book/shap.html" target="_blank" rel="noreferrer">9.6 SHAP (SHapley Additive exPlanations) | Interpretable Machine Learning</a>&#160;<a href="#fnref:4" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:5"> <p><a href="https://libraries.io/pypi/deeplift" target="_blank" rel="noreferrer">deeplift 0.6.13.0 on PyPI - Libraries.io</a>&#160;<a href="#fnref:5" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:6"> <p><a href="https://towardsdatascience.com/explainable-neural-networks-recent-advancements-part-3-6a838d15f2fb" target="_blank" rel="noreferrer">Explainable Neural Networks: Recent Advancements, Part 3 | by G Roshan Lal | Towards Data Science</a>&#160;<a href="#fnref:6" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:7"> <p>Shrikumar, A., Greenside, P., Shcherbina, A. &amp; Kundaje, A. Not Just a Black Box: Learning Important Features Through Propagating Activation Differences. Preprint at <a href="https://doi.org/10.48550/arXiv.1605.01713" target="_blank" rel="noreferrer">https://doi.org/10.48550/arXiv.1605.01713</a> (2017).&#160;<a href="#fnref:7" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:8"> <p>Shrikumar, A., Greenside, P. &amp; Kundaje, A. Learning Important Features Through Propagating Activation Differences. Preprint at <a href="http://arxiv.org/abs/1704.02685" target="_blank" rel="noreferrer">http://arxiv.org/abs/1704.02685</a> (2019).&#160;<a href="#fnref:8" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> </ol> </div>

2023/1/6
阅读更多

About

<p>I am currently an engineer of Center for Artificial Intelligence In Medicine in a Hospital. My research focuses on three areas:</p> <ol> <li>Secondary analysis EHRs</li> <li>Wearable device data analysis</li> <li>Multimodal fusion learning in healthcare</li> </ol>

2022/11/3
阅读更多

Friends

<div class="lead !mb-9 text-xl"> 友链以及值得关注的大佬！ </div> <table> <thead> <tr> <th>Website</th> <th>Details</th> </tr> </thead> <tbody> <tr> <td><a href="https://leonis.cc/" target="_blank" rel="noreferrer">Leo&rsquo;s blog</a></td> <td>数间茅舍，藏书万卷，投老村家</td> </tr> <tr> <td><a href="https://irenechen.net" target="_blank" rel="noreferrer">Irene Y. Chen </a></td> <td>Irene study machine learning for equitable healthcare</td> </tr> </tbody> </table> <p><strong>want to be a friend?</strong> To add your site to this list, please fell free to email me</p>

2020/8/14
阅读更多

PyTorch深度学习（2）

<script> document.addEventListener("DOMContentLoaded", function() { renderMathInElement(document.body, { delimiters: [ {left: '$$', right: '$$', display: true}, {left: '$', right: '$', display: false}, {left: '\\(', right: '\\)', display: false}, {left: '\\[', right: '\\]', display: true} ], throwOnError : false }); }); </script> <div class="lead !mb-9 text-xl"> Deep Learning = Learning Hierarchical Representations 深度学习即学习层次的表征。 </div> <h2 id="1-卷积神经网络" class="relative group">1. 卷积神经网络 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#1-%e5%8d%b7%e7%a7%af%e7%a5%9e%e7%bb%8f%e7%bd%91%e7%bb%9c" aria-label="Anchor">#</a></span></h2><h3 id="11-神经网络可视化visualization-of-neural-networks" class="relative group">1.1 神经网络可视化（Visualization of neural networks） <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#11-%e7%a5%9e%e7%bb%8f%e7%bd%91%e7%bb%9c%e5%8f%af%e8%a7%86%e5%8c%96visualization-of-neural-networks" aria-label="Anchor">#</a></span></h3><p>神经网络每一层的操作有点像将空间某些区域进行折叠</p> <h3 id="12-卷积神经网络的起源convolutional-neural-networkcnn" class="relative group">1.2 卷积神经网络的起源（Convolutional Neural Network；CNN） <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#12-%e5%8d%b7%e7%a7%af%e7%a5%9e%e7%bb%8f%e7%bd%91%e7%bb%9c%e7%9a%84%e8%b5%b7%e6%ba%90convolutional-neural-networkcnn" aria-label="Anchor">#</a></span></h3><p>受到Fukushima在视觉皮层建模方面的启发，使用简单/复杂的细胞层次结构，结合有监督的训练和反向传播，由Yann LeCun教授于88-89年在多伦多大学开发了第一个CNN。</p> <p>Fukushima的工作具体是什么呢？<br> 手写数字识别。首次提出应用多层简单或者复杂的细胞结构建模，特征：手工加无监督聚类学习。无反向传播。</p> <h3 id="13-卷积神经网络分解" class="relative group">1.3 卷积神经网络分解 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#13-%e5%8d%b7%e7%a7%af%e7%a5%9e%e7%bb%8f%e7%bd%91%e7%bb%9c%e5%88%86%e8%a7%a3" aria-label="Anchor">#</a></span></h3><p>通用的CNN架构能被分解为以下几个基本结构。</p> <ul> <li>标准化（Normalisation）:对比度标准化等</li> <li>滤波器组（Filter banks）:边缘检测等</li> <li>非线性化（Non-linearities）:稀疏化、ReLU等</li> <li>池化（pooling）:最大池化（max pooling）等</li> </ul> <h2 id="2-自然信号数据natural-signals" class="relative group">2. 自然信号数据（Natural Signals） <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#2-%e8%87%aa%e7%84%b6%e4%bf%a1%e5%8f%b7%e6%95%b0%e6%8d%aenatural-signals" aria-label="Anchor">#</a></span></h2><h3 id="21-自然信号数据特性" class="relative group">2.1 自然信号数据特性 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#21-%e8%87%aa%e7%84%b6%e4%bf%a1%e5%8f%b7%e6%95%b0%e6%8d%ae%e7%89%b9%e6%80%a7" aria-label="Anchor">#</a></span></h3><ul> <li>周期性：在时域很多模式都会重复出现</li> <li>局部性：相邻的点较相远的点来说更具关联性</li> <li>合成性：复杂的事物可以由简单的事物组合而成。字母-&gt;单词-&gt;句子-&gt;文章</li> </ul> <h3 id="22-对应神经网络中的处理方法" class="relative group">2.2 对应神经网络中的处理方法 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#22-%e5%af%b9%e5%ba%94%e7%a5%9e%e7%bb%8f%e7%bd%91%e7%bb%9c%e4%b8%ad%e7%9a%84%e5%a4%84%e7%90%86%e6%96%b9%e6%b3%95" aria-label="Anchor">#</a></span></h3><ul> <li>周期性$\rightarrow$参数共享<br> 如果数据存在周期性，可以使用参数共享，即卷积核。</li> <li>局部性$\rightarrow$稀疏<br> 如果数据存在局部性，那么每个神经元只需要与前几个神经元连接</li> <li>合成性$\rightarrow$多层<br> 即神经网络中多层网络合成最终的结果</li> </ul> <h2 id="3-pytorch实现mnist手写字识别" class="relative group">3. Pytorch实现Mnist手写字识别 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#3-pytorch%e5%ae%9e%e7%8e%b0mnist%e6%89%8b%e5%86%99%e5%ad%97%e8%af%86%e5%88%ab" aria-label="Anchor">#</a></span></h2><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="c1"># load package and data</span> </span></span><span class="line"><span class="cl"><span class="kn">import</span> <span class="nn">torch</span> </span></span><span class="line"><span class="cl"><span class="kn">import</span> <span class="nn">torch.nn</span> <span class="k">as</span> <span class="nn">nn</span> </span></span><span class="line"><span class="cl"><span class="kn">import</span> <span class="nn">torch.nn.functional</span> <span class="k">as</span> <span class="nn">F</span> </span></span><span class="line"><span class="cl"><span class="kn">import</span> <span class="nn">torch.optim</span> <span class="k">as</span> <span class="nn">optim</span> </span></span><span class="line"><span class="cl"><span class="kn">from</span> <span class="nn">torchvision</span> <span class="kn">import</span> <span class="n">datasets</span><span class="p">,</span> <span class="n">transforms</span> </span></span><span class="line"><span class="cl"><span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="nn">plt</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="n">device</span> <span class="o">=</span> <span class="n">torch</span><span class="o">.</span><span class="n">device</span><span class="p">(</span><span class="s2">&#34;cuda:0&#34;</span> <span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">cuda</span><span class="o">.</span><span class="n">is_available</span><span class="p">()</span> <span class="k">else</span> <span class="s2">&#34;cpu&#34;</span><span class="p">)</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="c1"># 神经网络模型偏爱标准化数据，原因是均值为0方差为1的数据在sigmoid、tanh经过激活函数后求导得到的导数很大，</span> </span></span><span class="line"><span class="cl"><span class="c1"># 反之原始数据不仅分布不均（噪声大）而且数值通常都很大（本例中数值范围是0~255），激活函数后求导得到的导数</span> </span></span><span class="line"><span class="cl"><span class="c1"># 则接近与0，这也被称为梯度消失。</span> </span></span><span class="line"><span class="cl"><span class="c1"># 目录放自己下载好的mnist目录，没有下载将download=True,自己新建一个存放数据目录即可</span> </span></span><span class="line"><span class="cl"><span class="n">train_loader</span> <span class="o">=</span> <span class="n">torch</span><span class="o">.</span><span class="n">utils</span><span class="o">.</span><span class="n">data</span><span class="o">.</span><span class="n">DataLoader</span><span class="p">(</span> </span></span><span class="line"><span class="cl"> <span class="n">datasets</span><span class="o">.</span><span class="n">MNIST</span><span class="p">(</span><span class="s1">&#39;../LSTM_mnist/mnist&#39;</span><span class="p">,</span> <span class="n">train</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">download</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> </span></span><span class="line"><span class="cl"> <span class="n">transform</span><span class="o">=</span><span class="n">transforms</span><span class="o">.</span><span class="n">Compose</span><span class="p">([</span> </span></span><span class="line"><span class="cl"> <span class="n">transforms</span><span class="o">.</span><span class="n">ToTensor</span><span class="p">(),</span> </span></span><span class="line"><span class="cl"> <span class="c1"># mnist数据集均值0.1307，标准差0.3081</span> </span></span><span class="line"><span class="cl"> <span class="n">transforms</span><span class="o">.</span><span class="n">Normalize</span><span class="p">((</span><span class="mf">0.1307</span><span class="p">,),</span> <span class="p">(</span><span class="mf">0.3081</span><span class="p">,))</span> </span></span><span class="line"><span class="cl"> <span class="p">])),</span> </span></span><span class="line"><span class="cl"> <span class="n">batch_size</span><span class="o">=</span><span class="mi">64</span><span class="p">,</span> <span class="n">shuffle</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="n">test_loader</span> <span class="o">=</span> <span class="n">torch</span><span class="o">.</span><span class="n">utils</span><span class="o">.</span><span class="n">data</span><span class="o">.</span><span class="n">DataLoader</span><span class="p">(</span> </span></span><span class="line"><span class="cl"> <span class="n">datasets</span><span class="o">.</span><span class="n">MNIST</span><span class="p">(</span><span class="s1">&#39;../LSTM_mnist/mnist&#39;</span><span class="p">,</span> <span class="n">train</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">transform</span><span class="o">=</span><span class="n">transforms</span><span class="o">.</span><span class="n">Compose</span><span class="p">([</span> </span></span><span class="line"><span class="cl"> <span class="n">transforms</span><span class="o">.</span><span class="n">ToTensor</span><span class="p">(),</span> </span></span><span class="line"><span class="cl"> <span class="n">transforms</span><span class="o">.</span><span class="n">Normalize</span><span class="p">((</span><span class="mf">0.1307</span><span class="p">,),</span> <span class="p">(</span><span class="mf">0.3081</span><span class="p">,))</span> </span></span><span class="line"><span class="cl"> <span class="p">])),</span> </span></span><span class="line"><span class="cl"> <span class="n">batch_size</span><span class="o">=</span><span class="mi">1000</span><span class="p">,</span> <span class="n">shuffle</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span> </span></span></code></pre></div><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="c1"># define model</span> </span></span><span class="line"><span class="cl"><span class="k">class</span> <span class="nc">SimpleCNN</span><span class="p">(</span><span class="n">nn</span><span class="o">.</span><span class="n">Module</span><span class="p">):</span> </span></span><span class="line"><span class="cl"> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">input_size</span><span class="p">,</span> <span class="n">n_feature</span><span class="p">,</span> <span class="n">output_size</span><span class="p">):</span> </span></span><span class="line"><span class="cl"> <span class="nb">super</span><span class="p">(</span><span class="n">SimpleCNN</span><span class="p">,</span> <span class="bp">self</span><span class="p">)</span><span class="o">.</span><span class="fm">__init__</span><span class="p">()</span> </span></span><span class="line"><span class="cl"> <span class="bp">self</span><span class="o">.</span><span class="n">n_feature</span> <span class="o">=</span> <span class="n">n_feature</span> </span></span><span class="line"><span class="cl"> <span class="c1"># 关于nn.Conv2d()中参数的解释</span> </span></span><span class="line"><span class="cl"> <span class="c1"># in_channels (int): Number of channels in the input image</span> </span></span><span class="line"><span class="cl"> <span class="c1"># out_channels (int): Number of channels produced by the convolution</span> </span></span><span class="line"><span class="cl"> <span class="c1"># kernel_size (int or tuple): Size of the convolving kernel</span> </span></span><span class="line"><span class="cl"> <span class="c1"># default stride=1, padding=0, dilation=1, groups=1</span> </span></span><span class="line"><span class="cl"> <span class="c1"># [groupsc参数详解](https://www.jianshu.com/p/20ba3d8f283c)</span> </span></span><span class="line"><span class="cl"> <span class="c1"># [图解卷积神经网络中stride, padding等操作可视化](https://github.com/vdumoulin/conv_arithmetic)</span> </span></span><span class="line"><span class="cl"> <span class="c1"># input: (N, C_in, H_in, W_in)</span> </span></span><span class="line"><span class="cl"> <span class="bp">self</span><span class="o">.</span><span class="n">conv1</span> <span class="o">=</span> <span class="n">nn</span><span class="o">.</span><span class="n">Conv2d</span><span class="p">(</span><span class="n">in_channels</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">out_channels</span><span class="o">=</span><span class="n">n_feature</span><span class="p">,</span> <span class="n">kernel_size</span><span class="o">=</span><span class="mi">5</span><span class="p">)</span> </span></span><span class="line"><span class="cl"> <span class="bp">self</span><span class="o">.</span><span class="n">conv2</span> <span class="o">=</span> <span class="n">nn</span><span class="o">.</span><span class="n">Conv2d</span><span class="p">(</span><span class="n">n_feature</span><span class="p">,</span> <span class="n">n_feature</span><span class="p">,</span> <span class="n">kernel_size</span><span class="o">=</span><span class="mi">5</span><span class="p">)</span> </span></span><span class="line"><span class="cl"> <span class="bp">self</span><span class="o">.</span><span class="n">fc1</span> <span class="o">=</span> <span class="n">nn</span><span class="o">.</span><span class="n">Linear</span><span class="p">(</span><span class="n">n_feature</span><span class="o">*</span><span class="mi">4</span><span class="o">*</span><span class="mi">4</span><span class="p">,</span> <span class="mi">50</span><span class="p">)</span> </span></span><span class="line"><span class="cl"> <span class="bp">self</span><span class="o">.</span><span class="n">fc2</span> <span class="o">=</span> <span class="n">nn</span><span class="o">.</span><span class="n">Linear</span><span class="p">(</span><span class="mi">50</span><span class="p">,</span> <span class="mi">10</span><span class="p">)</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"> <span class="k">def</span> <span class="nf">forward</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">x</span><span class="p">):</span> </span></span><span class="line"><span class="cl"> <span class="n">x</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">conv1</span><span class="p">(</span><span class="n">x</span><span class="p">)</span> <span class="c1"># Mnist数据原始大小（28*28）28-5+1 = 24 (24*24*n_feature)</span> </span></span><span class="line"><span class="cl"> <span class="n">x</span> <span class="o">=</span> <span class="n">F</span><span class="o">.</span><span class="n">relu</span><span class="p">(</span><span class="n">x</span><span class="p">)</span> </span></span><span class="line"><span class="cl"> <span class="n">x</span> <span class="o">=</span> <span class="n">F</span><span class="o">.</span><span class="n">max_pool2d</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">kernel_size</span><span class="o">=</span><span class="mi">2</span><span class="p">)</span> <span class="c1"># (12*12*n_feature)</span> </span></span><span class="line"><span class="cl"> <span class="n">x</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">conv2</span><span class="p">(</span><span class="n">x</span><span class="p">)</span> <span class="c1"># 12-5+1 = 8 (8*8*n_feature)</span> </span></span><span class="line"><span class="cl"> <span class="n">x</span> <span class="o">=</span> <span class="n">F</span><span class="o">.</span><span class="n">relu</span><span class="p">(</span><span class="n">x</span><span class="p">)</span> </span></span><span class="line"><span class="cl"> <span class="n">x</span> <span class="o">=</span> <span class="n">F</span><span class="o">.</span><span class="n">max_pool2d</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">kernel_size</span><span class="o">=</span><span class="mi">2</span><span class="p">)</span> <span class="c1"># (4*4*n_feature)这里解释了上面全连接时为啥是4*4</span> </span></span><span class="line"><span class="cl"> <span class="n">x</span> <span class="o">=</span> <span class="n">x</span><span class="o">.</span><span class="n">view</span><span class="p">(</span><span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">n_feature</span><span class="o">*</span><span class="mi">4</span><span class="o">*</span><span class="mi">4</span><span class="p">)</span> </span></span><span class="line"><span class="cl"> <span class="n">x</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">fc1</span><span class="p">(</span><span class="n">x</span><span class="p">)</span> </span></span><span class="line"><span class="cl"> <span class="n">x</span> <span class="o">=</span> <span class="n">F</span><span class="o">.</span><span class="n">relu</span><span class="p">(</span><span class="n">x</span><span class="p">)</span> </span></span><span class="line"><span class="cl"> <span class="n">x</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">fc2</span><span class="p">(</span><span class="n">x</span><span class="p">)</span> </span></span><span class="line"><span class="cl"> <span class="n">x</span> <span class="o">=</span> <span class="n">F</span><span class="o">.</span><span class="n">log_softmax</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">dim</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span> </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="n">x</span> </span></span></code></pre></div><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="c1"># hyper parameters</span> </span></span><span class="line"><span class="cl"><span class="n">epochs</span> <span class="o">=</span> <span class="mi">1</span> </span></span><span class="line"><span class="cl"><span class="n">input_size</span> <span class="o">=</span> <span class="mi">28</span><span class="o">*</span><span class="mi">28</span> </span></span><span class="line"><span class="cl"><span class="n">output_size</span> <span class="o">=</span> <span class="mi">10</span> </span></span><span class="line"><span class="cl"><span class="n">n_features</span> <span class="o">=</span> <span class="mi">6</span> </span></span><span class="line"><span class="cl"><span class="n">lr</span> <span class="o">=</span> <span class="mf">0.01</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="n">model</span> <span class="o">=</span> <span class="n">SimpleCNN</span><span class="p">(</span><span class="n">input_size</span><span class="p">,</span> <span class="n">n_features</span><span class="p">,</span> <span class="n">output_size</span><span class="p">)</span> </span></span><span class="line"><span class="cl"><span class="n">model</span><span class="o">.</span><span class="n">to</span><span class="p">(</span><span class="n">device</span><span class="p">)</span> </span></span><span class="line"><span class="cl"><span class="c1"># optimizer</span> </span></span><span class="line"><span class="cl"><span class="n">optimizer</span> <span class="o">=</span> <span class="n">optim</span><span class="o">.</span><span class="n">SGD</span><span class="p">(</span><span class="n">model</span><span class="o">.</span><span class="n">parameters</span><span class="p">(),</span> <span class="n">lr</span><span class="p">,</span> <span class="n">momentum</span><span class="o">=</span><span class="mf">0.5</span><span class="p">)</span> </span></span><span class="line"><span class="cl"><span class="nb">print</span><span class="p">(</span><span class="s1">&#39;Number of parameters: </span><span class="si">{}</span><span class="s1">&#39;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">get_n_params</span><span class="p">(</span><span class="n">model</span><span class="p">)))</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="c1"># model train</span> </span></span><span class="line"><span class="cl"><span class="k">for</span> <span class="n">epoch</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">epochs</span><span class="p">):</span> </span></span><span class="line"><span class="cl"> <span class="n">model</span><span class="o">.</span><span class="n">train</span><span class="p">()</span> </span></span><span class="line"><span class="cl"> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="n">target</span><span class="p">)</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">train_loader</span><span class="p">):</span> </span></span><span class="line"><span class="cl"> <span class="n">data</span><span class="p">,</span> <span class="n">target</span> <span class="o">=</span> <span class="n">data</span><span class="o">.</span><span class="n">to</span><span class="p">(</span><span class="n">device</span><span class="p">),</span> <span class="n">target</span><span class="o">.</span><span class="n">to</span><span class="p">(</span><span class="n">device</span><span class="p">)</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"> <span class="n">output</span> <span class="o">=</span> <span class="n">model</span><span class="p">(</span><span class="n">data</span><span class="p">)</span> </span></span><span class="line"><span class="cl"> <span class="n">loss</span> <span class="o">=</span> <span class="n">F</span><span class="o">.</span><span class="n">nll_loss</span><span class="p">(</span><span class="n">output</span><span class="p">,</span> <span class="n">target</span><span class="p">)</span> </span></span><span class="line"><span class="cl"> <span class="n">optimizer</span><span class="o">.</span><span class="n">zero_grad</span><span class="p">()</span> </span></span><span class="line"><span class="cl"> <span class="n">loss</span><span class="o">.</span><span class="n">backward</span><span class="p">()</span> </span></span><span class="line"><span class="cl"> <span class="n">optimizer</span><span class="o">.</span><span class="n">step</span><span class="p">()</span> </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="n">i</span> <span class="o">%</span> <span class="mi">100</span> <span class="o">==</span> <span class="mi">0</span><span class="p">:</span> </span></span><span class="line"><span class="cl"> <span class="nb">print</span><span class="p">(</span><span class="s1">&#39;Train Epoch [</span><span class="si">{}</span><span class="s1">/</span><span class="si">{}</span><span class="s1">], [</span><span class="si">{}</span><span class="s1">/</span><span class="si">{}</span><span class="s1"> (</span><span class="si">{:.0f}</span><span class="s1">%)], Loss: </span><span class="si">{:.4f}</span><span class="s1">&#39;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span> </span></span><span class="line"><span class="cl"> <span class="n">epoch</span><span class="o">+</span><span class="mi">1</span><span class="p">,</span> <span class="n">epochs</span><span class="p">,</span> <span class="n">i</span><span class="o">*</span><span class="nb">len</span><span class="p">(</span><span class="n">data</span><span class="p">),</span> <span class="nb">len</span><span class="p">(</span><span class="n">train_loader</span><span class="o">.</span><span class="n">dataset</span><span class="p">),</span> </span></span><span class="line"><span class="cl"> <span class="mi">100</span><span class="o">*</span><span class="n">i</span><span class="o">/</span><span class="nb">len</span><span class="p">(</span><span class="n">train_loader</span><span class="p">),</span> <span class="n">loss</span><span class="o">.</span><span class="n">item</span><span class="p">()))</span> </span></span><span class="line"><span class="cl"><span class="c1"># model eval</span> </span></span><span class="line"><span class="cl"><span class="n">model</span><span class="o">.</span><span class="n">eval</span><span class="p">()</span> </span></span><span class="line"><span class="cl"><span class="n">test_loss</span> <span class="o">=</span> <span class="mi">0</span> </span></span><span class="line"><span class="cl"><span class="n">correct</span> <span class="o">=</span> <span class="mi">0</span> </span></span><span class="line"><span class="cl"><span class="n">accuracy_list</span> <span class="o">=</span> <span class="p">[]</span> </span></span><span class="line"><span class="cl"><span class="k">for</span> <span class="n">data</span><span class="p">,</span> <span class="n">target</span> <span class="ow">in</span> <span class="n">test_loader</span><span class="p">:</span> </span></span><span class="line"><span class="cl"> <span class="n">data</span><span class="p">,</span> <span class="n">target</span> <span class="o">=</span> <span class="n">data</span><span class="o">.</span><span class="n">to</span><span class="p">(</span><span class="n">device</span><span class="p">),</span> <span class="n">target</span><span class="o">.</span><span class="n">to</span><span class="p">(</span><span class="n">device</span><span class="p">)</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"> <span class="n">output</span> <span class="o">=</span> <span class="n">model</span><span class="p">(</span><span class="n">data</span><span class="p">)</span> </span></span><span class="line"><span class="cl"> <span class="n">test_loss</span> <span class="o">+=</span> <span class="n">F</span><span class="o">.</span><span class="n">nll_loss</span><span class="p">(</span><span class="n">output</span><span class="p">,</span> <span class="n">target</span><span class="p">,</span> <span class="n">reduction</span><span class="o">=</span><span class="s1">&#39;sum&#39;</span><span class="p">)</span><span class="o">.</span><span class="n">item</span><span class="p">()</span> </span></span><span class="line"><span class="cl"> <span class="n">pred</span> <span class="o">=</span> <span class="n">output</span><span class="o">.</span><span class="n">data</span><span class="o">.</span><span class="n">max</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="n">keepdim</span><span class="o">=</span><span class="kc">True</span><span class="p">)[</span><span class="mi">1</span><span class="p">]</span> <span class="c1"># get the index of the max log-probability</span> </span></span><span class="line"><span class="cl"> <span class="n">correct</span> <span class="o">+=</span> <span class="n">pred</span><span class="o">.</span><span class="n">eq</span><span class="p">(</span><span class="n">target</span><span class="o">.</span><span class="n">data</span><span class="o">.</span><span class="n">view_as</span><span class="p">(</span><span class="n">pred</span><span class="p">))</span><span class="o">.</span><span class="n">cpu</span><span class="p">()</span><span class="o">.</span><span class="n">sum</span><span class="p">()</span><span class="o">.</span><span class="n">item</span><span class="p">()</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="n">test_loss</span> <span class="o">/=</span> <span class="nb">len</span><span class="p">(</span><span class="n">test_loader</span><span class="o">.</span><span class="n">dataset</span><span class="p">)</span> </span></span><span class="line"><span class="cl"><span class="n">accuracy</span> <span class="o">=</span> <span class="mf">100.</span> <span class="o">*</span> <span class="n">correct</span> <span class="o">/</span> <span class="nb">len</span><span class="p">(</span><span class="n">test_loader</span><span class="o">.</span><span class="n">dataset</span><span class="p">)</span> </span></span><span class="line"><span class="cl"><span class="n">accuracy_list</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">accuracy</span><span class="p">)</span> </span></span><span class="line"><span class="cl"><span class="nb">print</span><span class="p">(</span><span class="s1">&#39;</span><span class="se">\n</span><span class="s1">Test set Average loss: </span><span class="si">{:.4f}</span><span class="s1">, Accuracy: </span><span class="si">{}</span><span class="s1">/</span><span class="si">{}</span><span class="s1"> (</span><span class="si">{:.0f}</span><span class="s1">%)</span><span class="se">\n</span><span class="s1">&#39;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span> </span></span><span class="line"><span class="cl"> <span class="n">test_loss</span><span class="p">,</span> <span class="n">correct</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">test_loader</span><span class="o">.</span><span class="n">dataset</span><span class="p">),</span> <span class="n">accuracy</span><span class="p">))</span> </span></span></code></pre></div><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="c1"># 几个预测的实例可视化</span> </span></span><span class="line"><span class="cl"><span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">):</span> </span></span><span class="line"><span class="cl"> <span class="n">plt</span><span class="o">.</span><span class="n">subplot</span><span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="n">i</span><span class="o">+</span><span class="mi">1</span><span class="p">)</span> </span></span><span class="line"><span class="cl"> <span class="n">plt</span><span class="o">.</span><span class="n">imshow</span><span class="p">(</span><span class="n">data</span><span class="p">[</span><span class="n">i</span><span class="p">][</span><span class="mi">0</span><span class="p">])</span> </span></span><span class="line"><span class="cl"> <span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s2">&#34;Prediction: </span><span class="si">{}</span><span class="s2">&#34;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span> </span></span><span class="line"><span class="cl"> <span class="n">output</span><span class="o">.</span><span class="n">data</span><span class="o">.</span><span class="n">max</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="n">keepdim</span><span class="o">=</span><span class="kc">True</span><span class="p">)[</span><span class="mi">1</span><span class="p">][</span><span class="n">i</span><span class="p">]</span><span class="o">.</span><span class="n">item</span><span class="p">()))</span> </span></span><span class="line"><span class="cl"><span class="n">plt</span><span class="o">.</span><span class="n">show</span><span class="p">()</span> </span></span></code></pre></div><p> <figure><img src="https://youngforever.tech/images/pytorch/mnist_pre.png" alt="" class="mx-auto my-0 rounded-md" /> </figure> </p> <h2 id="4-补充" class="relative group">4. 补充 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#4-%e8%a1%a5%e5%85%85" aria-label="Anchor">#</a></span></h2><ol> <li>可以做一个有趣的实验即打乱图片中的像素后CNN识别正确率下降，而全连接网络则不会，即与最开始提到的三个特性以及对于神经网络采取的假设是吻合的。</li> <li>参考2中是对卷积神经网络全面的介绍，包括CNN中常用那些层，以及常用的模型和参数多少计算。</li> </ol> <h2 id="ref" class="relative group">ref <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#ref" aria-label="Anchor">#</a></span></h2><ol> <li><a href="https://atcold.github.io/pytorch-Deep-Learning/" target="_blank" rel="noreferrer">NYC PyTorch Deep Learning</a>课程网站</li> <li>cs231n <a href="https://cs231n.github.io/convolutional-networks/" target="_blank" rel="noreferrer">convolutional networks</a></li> <li><a href="https://pytorch.org/docs/stable/nn.html#convolution-layers" target="_blank" rel="noreferrer">pytorch官方文档Conv2d</a></li> <li><a href="https://github.com/Atcold/pytorch-Deep-Learning/blob/master/06-convnet.ipynb" target="_blank" rel="noreferrer">课程convnet.ipynb</a></li> </ol>

2020/4/1
阅读更多

重读XGBoost

<script> document.addEventListener("DOMContentLoaded", function() { renderMathInElement(document.body, { delimiters: [ {left: '$$', right: '$$', display: true}, {left: '$', right: '$', display: false}, {left: '\\(', right: '\\)', display: false}, {left: '\\[', right: '\\]', display: true} ], throwOnError : false }); }); </script> <p>在使用xgboost方法调参时，对其中个别参数不是特别理解。故重新读了一遍原论文。</p> <h2 id="1-引言" class="relative group">1. 引言 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#1-%e5%bc%95%e8%a8%80" aria-label="Anchor">#</a></span></h2><p>阐述机器学习和数据驱动的方法应用时两个重要的因素：</p> <ul> <li>能捕捉数据间复杂依赖关系的模型</li> <li>可扩展的学习系统，可以从大量数据中学习</li> </ul> <p>在目前常用的方法中，梯度提升树（gradient tree boosting）在许多场景中效果都不错，作者列举了一些。提出xgboost方法在比赛以及各类问题中的应用。</p> <p>叙述XGBoost的优点：运行更快、拓展性更好。创新点包括：</p> <ul> <li>高度可拓展的端到端提升树（tree boosting）系统</li> <li>用于高效计算的加权分位数图（weighted quantile sketch）</li> <li>新颖的稀疏感知算法（sparsity-aware algorithm），用于并行树学习</li> <li>有效的缓存优化以及块（cache-aware block）结构用于外存（out-of-core）树学习 关于以上几点在正文中详解。</li> </ul> <p>论文结构：</p> <ol> <li>提升树（tree boosting）简介以及目标函数正则化</li> <li>分裂点寻找的方法</li> <li>系统设计，包括为每个优化提供量化支持的结果</li> <li>相关工作</li> <li>详细的端到端评估</li> <li>总结</li> </ol> <h2 id="2-提升树tree-boosting简介" class="relative group">2. 提升树（Tree Boosting）简介 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#2-%e6%8f%90%e5%8d%87%e6%a0%91tree-boosting%e7%ae%80%e4%bb%8b" aria-label="Anchor">#</a></span></h2><p>首先需要了解CART（Classification And Regression Tree）算法，对于cart分类树和回归树分别采用了：<code>Gini系数</code>、<code>和方差</code>度量方式来划分节点[1]。例如回归树，对于划分特征A, 划分点s使两边数据集D1和D2,求出使 D1和D2各自集合的均方差最小，同时D1和D2的均方差之和最小。 $$\underline{min}_{\text{A,s}}[\underline{min}_{\text{c1}}\sum_{x_i\in{D1(A,s)}}(y_i - c1)^2+\underline{min}_{\text{c2}}\sum_{x_i\in{D2(A,s)}}(y_i - c2)^2]$$ 其中，c1为D1的样本输出均值，c2为D2的样本输出均值, 回归树采用叶子节点的均值或者中位数来预测输出结果，$y_i$即样本的label，此时的输出值即下文中用到的$w_{q(x)}$。</p> <h3 id="21-正则化目标函数" class="relative group">2.1 正则化目标函数 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#21-%e6%ad%a3%e5%88%99%e5%8c%96%e7%9b%ae%e6%a0%87%e5%87%bd%e6%95%b0" aria-label="Anchor">#</a></span></h3><p>对于这种类型的集成树模型，用K棵树的结果来预测最后的结果（式1），那么问题来了我们怎么来求这些树的参数，每棵树都可以看做一个函数$f_i$包含树的结构以及最后叶节点权重，集成模型不像传统优化问题一样通过简单用梯度下降可以对所有的树进行学习求解，所以，在这里用到了加法策略，即固定已经学习到的，每次加一棵树来进行学习（式3）。 $$\hat{y_i} = \phi(x_i) = \sum_{k = 1}^{K} f_k(x_i)\tag1$$ 其中$f(x) = w_{q(x)}$，每个$f_k$对应一个独立的树结构q以及其叶节点权重w，为了学习模型中的参数，最小化下面正则化的目标函数。 $$L(\phi) = \sum_i l(\hat{y_i}, y_i) + \sum_k \Omega(f_k)\tag2$$</p> <p>$$\Omega(f_k) = \gamma T + \frac{1}{2}\lambda||w||^2$$ T是树的叶节点数</p> <h3 id="22-梯度提升树gradient-tree-boosting" class="relative group">2.2 梯度提升树（Gradient Tree Boosting） <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#22-%e6%a2%af%e5%ba%a6%e6%8f%90%e5%8d%87%e6%a0%91gradient-tree-boosting" aria-label="Anchor">#</a></span></h3><p>第t次预测值等于t-1加上第t棵树的结果 $$\hat{y_i} = \hat{y_i}^{t-1} + f_t(x_i)\tag3$$ 此时目标函数(式2)可以写成 $$L^{(t)} = \sum_{i=1} ^n l(y_i, \hat{y_i}^{(t-1)}+f_t(x_i)) + \Omega(f_t)\tag4$$ 该式子的二阶近似可以表达为(式5），可以参考补充中<a href="#taylor">二阶泰勒展开的一般形式</a> $$L^{(t)} \approx \sum_{i=1} ^n [l(y_i, \hat{y_i}^{(t-1)}) + g_if_t(x_i) + \frac{1}{2}h_if_t^2(x_i)]+\Omega(f_t)\tag5$$ 其中 $g_i=\partial_{\hat{y}^{(t-1)}}{l(y_i, \hat{y_i}^{(t-1)})}， h_i=\partial_{\hat{y}^{(t-1)}}^2{l(y_i, \hat{y_i}^{(t-1)})}$</p> <p>式5中去掉常数项，即label与第t-1次结果的损失函数，可以得到：</p> <p>$$\sum_{i=1}^n[g_if_t(x_i)+\frac{1}{2}h_if_t^2(x_i)]+\Omega(f_t)\tag6$$</p> <p>式6即对新树的优化目标函数，进一步合并可以写为：</p> <p>$$obj^{t} \approx \sum_{i=1}^{n}[g_i w_{q(x_i)} + \frac{1}{2}h_i w^2_{q(x_i)}] + \gamma T+ \frac{1}{2}\lambda \sum _{j=1}^T w_j^2$$</p> <p>定义$I_j = {i|q(x_i)=j}$即叶节点j上的实例。</p> <p>$$obj^{t} \approx \sum_{j=1}^T[(\sum_{i\in{I_j}}g_i)w_j+\frac{1}{2}(\sum_{i\in{I_j}}h_i+\lambda)w_j^2]+\gamma T\tag7$$</p> <p>上式7对$w_j$求导即可求出w最优值</p> <p>$$w_j^* = -\frac{\sum_{i\in{I_j}}g_i}{\sum_{i\in{I_j}}h_i+\lambda} $$</p> <p>令$G_j = \sum_{i\in{I_j}}g_i，H_j=\sum_{i\in{I_j}}h_i$</p> <p>此时，对应的最小值为: $$obj^* = -\frac{1}{2}\sum_{j=1}^T\frac{G_j^2}{H_j+\lambda}+\gamma T$$ $\color{red}\frac{G_j^2}{H_j+\lambda}$越大，loss越小，所以对叶节点进行分裂，分裂后增益定义为 $$Gain=\frac{1}{2}[\frac{G_L^2}{H_L+\lambda}+\frac{G_R^2}{H_R+\lambda}-\frac{(G_L+G_R)^2}{H_L+H_R+\lambda}]-\gamma\tag8$$</p> <h3 id="23-缩减和列抽样shrinkage-and-column-subsampling" class="relative group">2.3 缩减和列抽样（Shrinkage and Column Subsampling） <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#23-%e7%bc%a9%e5%87%8f%e5%92%8c%e5%88%97%e6%8a%bd%e6%a0%b7shrinkage-and-column-subsampling" aria-label="Anchor">#</a></span></h3><p>除了在目标函数中引入正则项，在防止过拟合方面xgboost还运用了两项技术，给每一步tree boosting得到的结果一个权重$\eta$，来降低每一步的影响从而给后面树的形成留下空间，比喻成优化问题中的学习率缩减；同时还用到随机森林中的列抽样，即随机特征筛选。</p> <h2 id="3-分裂点寻找算法" class="relative group">3. 分裂点寻找算法 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#3-%e5%88%86%e8%a3%82%e7%82%b9%e5%af%bb%e6%89%be%e7%ae%97%e6%b3%95" aria-label="Anchor">#</a></span></h2><h3 id="31-精确贪婪算法basic-exact-greedy-algorithm" class="relative group">3.1 精确贪婪算法（Basic Exact Greedy Algorithm） <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#31-%e7%b2%be%e7%a1%ae%e8%b4%aa%e5%a9%aa%e7%ae%97%e6%b3%95basic-exact-greedy-algorithm" aria-label="Anchor">#</a></span></h3><p>即按照2.2中式8来寻找分裂点 python<code>scikit-learn</code>，R<code>gbm</code>，单机的xgboost都支持。</p> <div align=center> <img src="https://youngforever.tech/images/xgboost/algorithm1.jpg" width=50% heigth=50% /> </div> <!-- ![Algorithm1](/images/xgboost/algorithm1.jpg) --> <h3 id="32-近似算法approximate-algorithm" class="relative group">3.2 近似算法（Approximate Algorithm） <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#32-%e8%bf%91%e4%bc%bc%e7%ae%97%e6%b3%95approximate-algorithm" aria-label="Anchor">#</a></span></h3><p>精确贪婪算法由于列举了所有可能的分裂点，在数据量很大不能全部写入内存时会导致不是那么高效。所以提出近似算法。对于每个特征，只考察分位点，减少计算复杂度。 近似算法存在两个变种：</p> <ul> <li>global: 学习每棵树前，提出候选分裂点</li> <li>local: 每次分裂前，重新提出候选分裂点</li> </ul> <div align=center> <img src="https://youngforever.tech/images/xgboost/algorithm2.jpg" width=50% heigth=50% /> </div> <!-- ![Algorithm2](/images/xgboost/algorithm2.jpg) --> <h3 id="33-加权分位数图weighted-quantile-sketch" class="relative group">3.3 加权分位数图（Weighted Quantile Sketch） <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#33-%e5%8a%a0%e6%9d%83%e5%88%86%e4%bd%8d%e6%95%b0%e5%9b%beweighted-quantile-sketch" aria-label="Anchor">#</a></span></h3><p>近似算法中最重要一点即提出候选分裂点，xgboost不是简单的按照样本个体进行分位，而是以损失函数二阶导数值作为权重进行分位数分裂。如何寻找二阶导数分位点，首先是利用权重计算排序函数，然后相邻相减值作为判断依据。问题是为什么会想到利用损失函数二阶导数值作为权重来划分。 文中给出式6可以变形为 $$\sum_{i=1}^n\frac{1}{2}h_i(f_t(x_i)-g_i/h_i)^2 + \Omega(f_t) + constant\tag9$$ 指出该式恰好是权重平方差损失函数，权重$h_i$以及label $g_i/h_i$ 自己从式6变不到式9，觉得中间符号是+还差不多。 看有人理解说变成式10才对。是否作者真的是这样想的，不得而知。欢迎指正。 $$\sum_{i=1}^n\frac{1}{2}h_i(f_t(x_i)-(-g_i/h_i))^2 + \Omega(f_t) + constant\tag{10}$$ <a href="https://datascience.stackexchange.com/questions/10997/need-help-understanding-xgboosts-approximate-split-points-proposal" target="_blank" rel="noreferrer">stackexchange上关于理解xgboost近似分裂点</a></p> <h3 id="34-稀疏值感知分裂sparsity-aware-split-finding" class="relative group">3.4 稀疏值感知分裂（Sparsity-aware split finding） <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#34-%e7%a8%80%e7%96%8f%e5%80%bc%e6%84%9f%e7%9f%a5%e5%88%86%e8%a3%82sparsity-aware-split-finding" aria-label="Anchor">#</a></span></h3><p>造成稀疏值的原因：1）缺失值 2）统计过程中频繁的0值输入 3）one-hot编码以及其他特征工程 所以让算法注意数据中稀疏规律很重要，遍历所有特征，在划分子节点时，统一将该特征的缺失值划分到右支或者左支，计算最大的gain。</p> <div align=center> <img src="https://youngforever.tech/images/xgboost/Sparsity.jpg" width=50% heigth=50% /> </div> <!-- ![Sparsity](/images/xgboost/Sparsity.jpg) --> <p>$\color{red}这里也有个疑问就是为什么排序第一次是升序，第二次是降序$</p> <h2 id="4-系统设计" class="relative group">4. 系统设计 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#4-%e7%b3%bb%e7%bb%9f%e8%ae%be%e8%ae%a1" aria-label="Anchor">#</a></span></h2><h3 id="41-分块并行column-block-for-parallel-learning" class="relative group">4.1 分块并行（Column Block for Parallel Learning） <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#41-%e5%88%86%e5%9d%97%e5%b9%b6%e8%a1%8ccolumn-block-for-parallel-learning" aria-label="Anchor">#</a></span></h3><p>基于树学习过程中最耗时的是将数据排序，为了减少排序的时间成本，提出基于内存的block结构。</p> <ul> <li>在Exact greedy算法中，将整个数据集存放在一个Block中</li> <li>在近似算法中，使用多个Block，每个Block对应原来数据的子集。不同的Block可以在不同的机器上并行计算</li> </ul> <h3 id="42-缓存优化" class="relative group">4.2 缓存优化 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#42-%e7%bc%93%e5%ad%98%e4%bc%98%e5%8c%96" aria-label="Anchor">#</a></span></h3><p>这里指利用CPU缓存对算法进行优化。</p> <p>4.1中column block按特征大小顺序存储，相应的样本的梯度信息是分散的，造成内存的不连续访问，降低CPU cache命中率。 优化方法：</p> <ul> <li>对于精确贪婪算法，预取数据到buffer中（非连续-&gt;连续），再统计梯度信息。</li> <li>对于近似算法，调节block的大小，设置过大则容易导致命中率低，过小则容易导致并行化效率不高。</li> </ul> <h3 id="43-外存计算" class="relative group">4.3 外存计算 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#43-%e5%a4%96%e5%ad%98%e8%ae%a1%e7%ae%97" aria-label="Anchor">#</a></span></h3><p>除了处理器以及内存，利用磁盘空间来处理不能进入内存的数据也十分重要，数据划分为多个Block并存放在磁盘上。计算的时候，使用独立的线程预先将Block放入主内存，因此可以在计算的同时读取磁盘。在减少计算资源开销以及提高磁盘输入输出方面主要用到以下技术：</p> <ul> <li>Block压缩，按列压缩，加载到主内存时由独立线程动态解压缩。具体压缩技术参看原文。</li> <li>Block Sharding，将数据划分到不同硬盘上，提高磁盘吞吐率。</li> </ul> <h2 id="5-端到端评估" class="relative group">5. 端到端评估 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#5-%e7%ab%af%e5%88%b0%e7%ab%af%e8%af%84%e4%bc%b0" aria-label="Anchor">#</a></span></h2><p>利用4个数据集对xgboost评估：</p> <ul> <li>分类问题</li> <li>排序问题</li> <li>外存计算实验</li> <li>分布计算实验</li> </ul> <p>这几个方面进行评估，详细结果见论文。</p> <h2 id="ref" class="relative group">ref <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#ref" aria-label="Anchor">#</a></span></h2><ol> <li><a href="https://blog.csdn.net/hy592070616/article/details/81628956" target="_blank" rel="noreferrer">CART分类树与回归树</a></li> <li><a href="https://www.cnblogs.com/xym4869/p/11282586.html" target="_blank" rel="noreferrer">Markdown数学公式</a></li> <li><a href="https://docs.mathjax.org/en/latest/web/configuration.html" target="_blank" rel="noreferrer">Mathjax应用在网页</a></li> <li><a href="https://homes.cs.washington.edu/~tqchen/pdf/BoostedTree.pdf" target="_blank" rel="noreferrer">XGBoost.ppt</a></li> <li><a href="https://xgboost.readthedocs.io/en/latest/tutorials/model.html" target="_blank" rel="noreferrer">readthedocs xgboost tutorials</a>推荐</li> <li><a href="http://wepon.me/files/gbdt.pdf" target="_blank" rel="noreferrer">gbdt.ppt</a></li> <li><a href="https://arxiv.org/abs/1603.02754" target="_blank" rel="noreferrer">xgboost原文</a></li> </ol> <h2 id="补充" class="relative group">补充 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e8%a1%a5%e5%85%85" aria-label="Anchor">#</a></span></h2><ol> <li>文中很多术语翻译可能有不恰当的地方，欢迎指出。</li> <li>二阶泰勒展开的一般形式： $$f(x^t) = f(x^{t-1}+\Delta x)\approx{f(x^{t-1})+ f^{\prime}(x^{t-1})\Delta{x}+f^{\prime\prime}(x^{t-1})\frac{\Delta x^2}{2}}$$</li> <li>式4中加入loss function是mean squared error(MSE)，可以求出相应的gi， hi作为一个特例来验证该做法。</li> <li>基于树的算法理解时带着这几个问题去理解每一步是用来做什么的：选择哪个特征进行分裂？在特征什么点位进行分裂？分裂后叶节点取什么值？ <blockquote> <p>分别对应：遍历每个特征，加权分位数图，$w_j$</p> </blockquote> </li> <li>对于系统设计中应用到的技术理解不是十分深刻，对应一个算法如何从计算机硬件的方方面面考虑去优化对非专业领域研究者还是比较难</li> </ol>

2020/3/23
阅读更多

LSTM应用场景以及pytorch实例

<script> document.addEventListener("DOMContentLoaded", function() { renderMathInElement(document.body, { delimiters: [ {left: '$$', right: '$$', display: true}, {left: '$', right: '$', display: false}, {left: '\\(', right: '\\)', display: false}, {left: '\\[', right: '\\]', display: true} ], throwOnError : false }); }); </script> <p>在去年介绍的一篇paper中，应用了多任务RNN来解决问题，当时RNN指的即是LSTM。本文介绍LSTM实现以及应用。</p> <h2 id="1-lstm简介" class="relative group">1. LSTM简介 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#1-lstm%e7%ae%80%e4%bb%8b" aria-label="Anchor">#</a></span></h2><p>循环神经网络要点在于可以将上一时刻的信息传递给下一时刻，但是在需要长程信息依赖的场景，训练一个好的RNN十分困难，存在梯度爆炸和梯度消失的情况。LSTM通过刻意的设计来解决该问题。</p> <p>简单的RNN网络中重复的模块只有一个简单的结构，例如一个<code>relu</code>层，而在LSTM中重复的模块拥有4个不同的结构相互交互来完成。</p> <p> <figure><img src="https://youngforever.tech/images/LSTM/LSTM.jpg" alt="LSTM" class="mx-auto my-0 rounded-md" /> </figure> </p> <h3 id="11-首先决定从cell中丢弃什么信息" class="relative group">1.1 首先决定从cell中丢弃什么信息 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#11-%e9%a6%96%e5%85%88%e5%86%b3%e5%ae%9a%e4%bb%8ecell%e4%b8%ad%e4%b8%a2%e5%bc%83%e4%bb%80%e4%b9%88%e4%bf%a1%e6%81%af" aria-label="Anchor">#</a></span></h3><p>$$f_t = \sigma(W_f*[h_{t-1}, X_t] + b_f) \tag1$$ sigma函数在0到1选择代表丢弃与否</p> <h3 id="12-什么样的新信息存放到cell中" class="relative group">1.2 什么样的新信息存放到cell中 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#12-%e4%bb%80%e4%b9%88%e6%a0%b7%e7%9a%84%e6%96%b0%e4%bf%a1%e6%81%af%e5%ad%98%e6%94%be%e5%88%b0cell%e4%b8%ad" aria-label="Anchor">#</a></span></h3><p>$$i_t = \sigma(W_i*[h_{t-1}, x_t] + b_i) \tag2$$</p> <p>$$\widetilde{C_t} = tanh(W_c*[h_{t-1}, x_t] + b_c) \tag3$$</p> <p>$$C_t = f_t*C_{t-1} + {i_t} * \widetilde{C_{t}} \tag4$$</p> <p>4式中旧状态与$f_t$相乘，丢弃确定需要丢弃的信息，加上新的候选值。可以看到假如遗忘门一直为1，就可以保持以前的信息$C_{t-1}$</p> <h3 id="13-输出结果" class="relative group">1.3 输出结果 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#13-%e8%be%93%e5%87%ba%e7%bb%93%e6%9e%9c" aria-label="Anchor">#</a></span></h3><p>$$o_t = \sigma(W_o[h_{t-1}, x_t] + b_o)\tag5$$ $$h_t = o_t*tanh(C_t)\tag6$$</p> <h2 id="2-lstm实例以及pytorch实现" class="relative group">2. LSTM实例以及Pytorch实现 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#2-lstm%e5%ae%9e%e4%be%8b%e4%bb%a5%e5%8f%8apytorch%e5%ae%9e%e7%8e%b0" aria-label="Anchor">#</a></span></h2><p>循环神经网络可以应用到以下场景。</p> <p> <figure><img src="https://youngforever.tech/images/LSTM/examples.jpg" alt="examples" class="mx-auto my-0 rounded-md" /> </figure> </p> <ul> <li>点对点（单个图片（文字）被分类；图像分类）</li> <li>点对序列（单个图像（文字）被分为多个类；图像输出文字）</li> <li>序列分析（一系列图片（文字）被分类；情感分析）</li> <li>不等长序列对序列（机器翻译）</li> <li>等长序列对序列（视频帧分类）</li> </ul> <p>举两个例子：图像分类以及时间序列预测</p> <h3 id="21-lstm图像分类" class="relative group">2.1 LSTM图像分类 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#21-lstm%e5%9b%be%e5%83%8f%e5%88%86%e7%b1%bb" aria-label="Anchor">#</a></span></h3><p>关于图片分类常用卷积神经网络，侧重空间上处理；而循环神经网络侧重序列处理。但是也能用来图片分类。第一个例子以常用的mnist手写字体识别为例。</p> <h4 id="211-导入所需用到的包以及超参数设置等" class="relative group">2.1.1 导入所需用到的包以及超参数设置等 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#211-%e5%af%bc%e5%85%a5%e6%89%80%e9%9c%80%e7%94%a8%e5%88%b0%e7%9a%84%e5%8c%85%e4%bb%a5%e5%8f%8a%e8%b6%85%e5%8f%82%e6%95%b0%e8%ae%be%e7%bd%ae%e7%ad%89" aria-label="Anchor">#</a></span></h4><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="c1"># Setup</span> </span></span><span class="line"><span class="cl"><span class="kn">import</span> <span class="nn">torch</span> </span></span><span class="line"><span class="cl"><span class="kn">from</span> <span class="nn">torch</span> <span class="kn">import</span> <span class="n">nn</span> </span></span><span class="line"><span class="cl"><span class="kn">from</span> <span class="nn">torch.utils.data</span> <span class="kn">import</span> <span class="n">DataLoader</span> </span></span><span class="line"><span class="cl"><span class="kn">import</span> <span class="nn">torchvision.datasets</span> <span class="k">as</span> <span class="nn">dsets</span> </span></span><span class="line"><span class="cl"><span class="kn">import</span> <span class="nn">torchvision.transforms</span> <span class="k">as</span> <span class="nn">transforms</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="n">torch</span><span class="o">.</span><span class="n">manual_seed</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="c1"># Device configuration</span> </span></span><span class="line"><span class="cl"><span class="n">device</span> <span class="o">=</span> <span class="n">torch</span><span class="o">.</span><span class="n">device</span><span class="p">(</span><span class="s1">&#39;cuda&#39;</span> <span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">cuda</span><span class="o">.</span><span class="n">is_available</span><span class="p">()</span> <span class="k">else</span> <span class="s1">&#39;cpu&#39;</span><span class="p">)</span> </span></span></code></pre></div><h4 id="212-导入数据集" class="relative group">2.1.2 导入数据集 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#212-%e5%af%bc%e5%85%a5%e6%95%b0%e6%8d%ae%e9%9b%86" aria-label="Anchor">#</a></span></h4><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="c1"># Mnist手写数字</span> </span></span><span class="line"><span class="cl"><span class="n">train_data</span> <span class="o">=</span> <span class="n">dsets</span><span class="o">.</span><span class="n">MNIST</span><span class="p">(</span><span class="n">root</span><span class="o">=</span><span class="s1">&#39;./mnist/&#39;</span><span class="p">,</span> <span class="c1"># 保存或者提取位置</span> </span></span><span class="line"><span class="cl"> <span class="n">train</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="c1"># this is tra`ining data</span> </span></span><span class="line"><span class="cl"> <span class="n">transform</span><span class="o">=</span><span class="n">transforms</span><span class="o">.</span><span class="n">ToTensor</span><span class="p">(),</span> <span class="c1"># 转换 PIL.Image or numpy.ndarray 成</span> </span></span><span class="line"><span class="cl"> <span class="c1"># torch.FloatTensor (C x H x W), 训练的时候 normalize 成 [0.0, 1.0] 区间</span> </span></span><span class="line"><span class="cl"> <span class="n">download</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="c1"># 没下载就下载, 下载了就不用再下了改成False</span> </span></span><span class="line"><span class="cl"><span class="p">)</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="n">test_data</span> <span class="o">=</span> <span class="n">dsets</span><span class="o">.</span><span class="n">MNIST</span><span class="p">(</span><span class="n">root</span><span class="o">=</span><span class="s1">&#39;./mnist/&#39;</span><span class="p">,</span> </span></span><span class="line"><span class="cl"> <span class="n">train</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> </span></span><span class="line"><span class="cl"> <span class="n">transform</span><span class="o">=</span><span class="n">transforms</span><span class="o">.</span><span class="n">ToTensor</span><span class="p">())</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="c1"># Dataloader</span> </span></span><span class="line"><span class="cl"><span class="c1"># PyTorch中数据读取的一个重要接口，该接口定义在dataloader.py中，只要是用PyTorch来训练模型基本都会用到该接口（除非用户重写…），</span> </span></span><span class="line"><span class="cl"><span class="c1"># 该接口的目的：将自定义的Dataset根据batch size大小、是否shuffle等封装成一个Batch Size大小的Tensor，用于后面的训练。</span> </span></span><span class="line"><span class="cl"><span class="n">train_loader</span> <span class="o">=</span> <span class="n">DataLoader</span><span class="p">(</span><span class="n">dataset</span><span class="o">=</span><span class="n">train_data</span><span class="p">,</span> </span></span><span class="line"><span class="cl"> <span class="n">batch_size</span><span class="o">=</span><span class="n">batch_size</span><span class="p">,</span> </span></span><span class="line"><span class="cl"> <span class="n">shuffle</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span> <span class="c1"># 在每个epoch开始的时候，对数据重新打乱进行训练。在这里其实没啥用，因为只训练了一次</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="n">test_loader</span> <span class="o">=</span> <span class="n">DataLoader</span><span class="p">(</span><span class="n">dataset</span><span class="o">=</span><span class="n">test_data</span><span class="p">,</span> </span></span><span class="line"><span class="cl"> <span class="n">batch_size</span><span class="o">=</span><span class="n">batch_size</span><span class="p">,</span> </span></span><span class="line"><span class="cl"> <span class="n">shuffle</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span> </span></span></code></pre></div><h4 id="213-建立模型" class="relative group">2.1.3 建立模型 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#213-%e5%bb%ba%e7%ab%8b%e6%a8%a1%e5%9e%8b" aria-label="Anchor">#</a></span></h4><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="c1"># LSTM</span> </span></span><span class="line"><span class="cl"><span class="c1"># __init__ is basically a function which will &#34;initialize&#34;/&#34;activate&#34; the properties of the class for a specific object</span> </span></span><span class="line"><span class="cl"><span class="c1"># self represents that object which will inherit those properties</span> </span></span><span class="line"><span class="cl"><span class="k">class</span> <span class="nc">simpleLSTM</span><span class="p">(</span><span class="n">nn</span><span class="o">.</span><span class="n">Module</span><span class="p">):</span> </span></span><span class="line"><span class="cl"> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">input_size</span><span class="p">,</span> <span class="n">hidden_size</span><span class="p">,</span> <span class="n">num_layers</span><span class="p">,</span> <span class="n">num_classes</span><span class="p">):</span> </span></span><span class="line"><span class="cl"> <span class="nb">super</span><span class="p">(</span><span class="n">simpleLSTM</span><span class="p">,</span> <span class="bp">self</span><span class="p">)</span><span class="o">.</span><span class="fm">__init__</span><span class="p">()</span> </span></span><span class="line"><span class="cl"> <span class="bp">self</span><span class="o">.</span><span class="n">hidden_size</span> <span class="o">=</span> <span class="n">hidden_size</span> </span></span><span class="line"><span class="cl"> <span class="bp">self</span><span class="o">.</span><span class="n">num_layers</span> <span class="o">=</span> <span class="n">num_layers</span> </span></span><span class="line"><span class="cl"> <span class="bp">self</span><span class="o">.</span><span class="n">lstm</span> <span class="o">=</span> <span class="n">nn</span><span class="o">.</span><span class="n">LSTM</span><span class="p">(</span><span class="n">input_size</span><span class="p">,</span> <span class="n">hidden_size</span><span class="p">,</span> <span class="n">num_layers</span><span class="p">,</span> <span class="n">batch_first</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span> </span></span><span class="line"><span class="cl"> <span class="bp">self</span><span class="o">.</span><span class="n">fc</span> <span class="o">=</span> <span class="n">nn</span><span class="o">.</span><span class="n">Linear</span><span class="p">(</span><span class="n">hidden_size</span><span class="p">,</span> <span class="n">num_classes</span><span class="p">)</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"> <span class="k">def</span> <span class="nf">forward</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">x</span><span class="p">):</span> </span></span><span class="line"><span class="cl"> <span class="c1"># x shape (batch, time_step, input_size)</span> </span></span><span class="line"><span class="cl"> <span class="c1"># out shape (batch, time_step, output_size)</span> </span></span><span class="line"><span class="cl"> <span class="c1"># h_n shape (n_layers, batch, hidden_size)</span> </span></span><span class="line"><span class="cl"> <span class="c1"># h_c shape (n_layers, batch, hidden_size)</span> </span></span><span class="line"><span class="cl"> <span class="c1"># 初始化hidden和memory cell参数</span> </span></span><span class="line"><span class="cl"> <span class="n">h0</span> <span class="o">=</span> <span class="n">torch</span><span class="o">.</span><span class="n">zeros</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">num_layers</span><span class="p">,</span> <span class="n">x</span><span class="o">.</span><span class="n">size</span><span class="p">(</span><span class="mi">0</span><span class="p">),</span> <span class="bp">self</span><span class="o">.</span><span class="n">hidden_size</span><span class="p">)</span><span class="o">.</span><span class="n">to</span><span class="p">(</span><span class="n">device</span><span class="p">)</span> </span></span><span class="line"><span class="cl"> <span class="n">c0</span> <span class="o">=</span> <span class="n">torch</span><span class="o">.</span><span class="n">zeros</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">num_layers</span><span class="p">,</span> <span class="n">x</span><span class="o">.</span><span class="n">size</span><span class="p">(</span><span class="mi">0</span><span class="p">),</span> <span class="bp">self</span><span class="o">.</span><span class="n">hidden_size</span><span class="p">)</span><span class="o">.</span><span class="n">to</span><span class="p">(</span><span class="n">device</span><span class="p">)</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"> <span class="c1"># forward propagate lstm</span> </span></span><span class="line"><span class="cl"> <span class="n">out</span><span class="p">,</span> <span class="p">(</span><span class="n">h_n</span><span class="p">,</span> <span class="n">h_c</span><span class="p">)</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">lstm</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="p">(</span><span class="n">h0</span><span class="p">,</span> <span class="n">c0</span><span class="p">))</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"> <span class="c1"># 选取最后一个时刻的输出</span> </span></span><span class="line"><span class="cl"> <span class="n">out</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">fc</span><span class="p">(</span><span class="n">out</span><span class="p">[:,</span> <span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="p">:])</span> </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="n">out</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="n">model</span> <span class="o">=</span> <span class="n">simpleLSTM</span><span class="p">(</span><span class="n">input_size</span><span class="p">,</span> <span class="n">hidden_size</span><span class="p">,</span> <span class="n">num_layers</span><span class="p">,</span> <span class="n">num_classes</span><span class="p">)</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="c1"># loss and optimizer</span> </span></span><span class="line"><span class="cl"><span class="n">criterion</span> <span class="o">=</span> <span class="n">nn</span><span class="o">.</span><span class="n">CrossEntropyLoss</span><span class="p">()</span> </span></span><span class="line"><span class="cl"><span class="n">optimizer</span> <span class="o">=</span> <span class="n">torch</span><span class="o">.</span><span class="n">optim</span><span class="o">.</span><span class="n">Adam</span><span class="p">(</span><span class="n">model</span><span class="o">.</span><span class="n">parameters</span><span class="p">(),</span> <span class="n">lr</span><span class="p">)</span> </span></span></code></pre></div><h4 id="214-训练模型" class="relative group">2.1.4 训练模型 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#214-%e8%ae%ad%e7%bb%83%e6%a8%a1%e5%9e%8b" aria-label="Anchor">#</a></span></h4><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="c1"># train the model</span> </span></span><span class="line"><span class="cl"><span class="c1"># 关于reshape(-1)的解释 https://www.zhihu.com/question/52684594</span> </span></span><span class="line"><span class="cl"><span class="c1"># view()和reshape()区别的解释 https://stackoverflow.com/questions/49643225/whats-the-difference-between-reshape-and-view-in-pytorch</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="c1"># Hyper Parameters</span> </span></span><span class="line"><span class="cl"><span class="n">epochs</span> <span class="o">=</span> <span class="mi">1</span> <span class="c1"># 训练整批数据多少次, 为了节约时间, 我们只训练一次</span> </span></span><span class="line"><span class="cl"><span class="n">batch_size</span> <span class="o">=</span> <span class="mi">64</span> </span></span><span class="line"><span class="cl"><span class="n">time_step</span> <span class="o">=</span> <span class="mi">28</span> <span class="c1"># rnn 时间步数 / 图片高度</span> </span></span><span class="line"><span class="cl"><span class="n">input_size</span> <span class="o">=</span> <span class="mi">28</span> <span class="c1"># rnn 每步输入值 / 图片每行像素</span> </span></span><span class="line"><span class="cl"><span class="n">hidden_size</span> <span class="o">=</span> <span class="mi">64</span> </span></span><span class="line"><span class="cl"><span class="n">num_layers</span> <span class="o">=</span> <span class="mi">1</span> </span></span><span class="line"><span class="cl"><span class="n">num_classes</span> <span class="o">=</span> <span class="mi">10</span> </span></span><span class="line"><span class="cl"><span class="n">lr</span> <span class="o">=</span> <span class="mf">0.01</span> <span class="c1"># learning rate</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="n">total_step</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="n">train_loader</span><span class="p">)</span> </span></span><span class="line"><span class="cl"><span class="k">for</span> <span class="n">epoch</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">epochs</span><span class="p">):</span> </span></span><span class="line"><span class="cl"> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="p">(</span><span class="n">images</span><span class="p">,</span> <span class="n">labels</span><span class="p">)</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">train_loader</span><span class="p">):</span> </span></span><span class="line"><span class="cl"> <span class="n">images</span> <span class="o">=</span> <span class="n">images</span><span class="o">.</span><span class="n">reshape</span><span class="p">(</span><span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="n">time_step</span><span class="p">,</span> <span class="n">input_size</span><span class="p">)</span><span class="o">.</span><span class="n">to</span><span class="p">(</span><span class="n">device</span><span class="p">)</span> </span></span><span class="line"><span class="cl"> <span class="n">labels</span> <span class="o">=</span> <span class="n">labels</span><span class="o">.</span><span class="n">to</span><span class="p">(</span><span class="n">device</span><span class="p">)</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"> <span class="c1"># forward pass</span> </span></span><span class="line"><span class="cl"> <span class="n">outputs</span> <span class="o">=</span> <span class="n">model</span><span class="p">(</span><span class="n">images</span><span class="p">)</span> </span></span><span class="line"><span class="cl"> <span class="n">loss</span> <span class="o">=</span> <span class="n">criterion</span><span class="p">(</span><span class="n">outputs</span><span class="p">,</span> <span class="n">labels</span><span class="p">)</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"> <span class="c1"># backward and optimize</span> </span></span><span class="line"><span class="cl"> <span class="n">optimizer</span><span class="o">.</span><span class="n">zero_grad</span><span class="p">()</span> </span></span><span class="line"><span class="cl"> <span class="n">loss</span><span class="o">.</span><span class="n">backward</span><span class="p">()</span> </span></span><span class="line"><span class="cl"> <span class="n">optimizer</span><span class="o">.</span><span class="n">step</span><span class="p">()</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="n">i</span> <span class="o">%</span> <span class="mi">100</span> <span class="o">==</span> <span class="mi">0</span><span class="p">:</span> </span></span><span class="line"><span class="cl"> <span class="nb">print</span><span class="p">(</span><span class="s1">&#39;Epoch [</span><span class="si">{}</span><span class="s1">/</span><span class="si">{}</span><span class="s1">], Step [</span><span class="si">{}</span><span class="s1">/</span><span class="si">{}</span><span class="s1">], Loss: </span><span class="si">{:.4f}</span><span class="s1">&#39;</span> </span></span><span class="line"><span class="cl"> <span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">epoch</span><span class="o">+</span><span class="mi">1</span><span class="p">,</span> <span class="n">epochs</span><span class="p">,</span> <span class="n">i</span><span class="o">+</span><span class="mi">1</span><span class="p">,</span> <span class="n">total_step</span><span class="p">,</span> <span class="n">loss</span><span class="o">.</span><span class="n">item</span><span class="p">()))</span> </span></span></code></pre></div><h4 id="215-测试模型" class="relative group">2.1.5 测试模型 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#215-%e6%b5%8b%e8%af%95%e6%a8%a1%e5%9e%8b" aria-label="Anchor">#</a></span></h4><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="c1"># Test the model</span> </span></span><span class="line"><span class="cl"><span class="c1"># https://stackoverflow.com/questions/55627780/evaluating-pytorch-models-with-torch-no-grad-vs-model-eval</span> </span></span><span class="line"><span class="cl"><span class="c1"># torch.max()用法。https://blog.csdn.net/weixin_43255962/article/details/84402586</span> </span></span><span class="line"><span class="cl"><span class="n">model</span><span class="o">.</span><span class="n">eval</span><span class="p">()</span> </span></span><span class="line"><span class="cl"><span class="k">with</span> <span class="n">torch</span><span class="o">.</span><span class="n">no_grad</span><span class="p">():</span> </span></span><span class="line"><span class="cl"> <span class="n">correct</span> <span class="o">=</span> <span class="mi">0</span> </span></span><span class="line"><span class="cl"> <span class="n">total</span> <span class="o">=</span> <span class="mi">0</span> </span></span><span class="line"><span class="cl"> <span class="k">for</span> <span class="n">images</span><span class="p">,</span> <span class="n">labels</span> <span class="ow">in</span> <span class="n">test_loader</span><span class="p">:</span> </span></span><span class="line"><span class="cl"> <span class="n">images</span> <span class="o">=</span> <span class="n">images</span><span class="o">.</span><span class="n">reshape</span><span class="p">(</span><span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="n">time_step</span><span class="p">,</span> <span class="n">input_size</span><span class="p">)</span><span class="o">.</span><span class="n">to</span><span class="p">(</span><span class="n">device</span><span class="p">)</span> </span></span><span class="line"><span class="cl"> <span class="n">labels</span> <span class="o">=</span> <span class="n">labels</span><span class="o">.</span><span class="n">to</span><span class="p">(</span><span class="n">device</span><span class="p">)</span> </span></span><span class="line"><span class="cl"> <span class="n">outputs</span> <span class="o">=</span> <span class="n">model</span><span class="p">(</span><span class="n">images</span><span class="p">)</span> </span></span><span class="line"><span class="cl"> <span class="n">_</span><span class="p">,</span> <span class="n">predicted</span> <span class="o">=</span> <span class="n">torch</span><span class="o">.</span><span class="n">max</span><span class="p">(</span><span class="n">outputs</span><span class="o">.</span><span class="n">data</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span> </span></span><span class="line"><span class="cl"> <span class="n">total</span> <span class="o">+=</span> <span class="n">labels</span><span class="o">.</span><span class="n">size</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span> </span></span><span class="line"><span class="cl"> <span class="n">correct</span> <span class="o">+=</span> <span class="p">(</span><span class="n">predicted</span> <span class="o">==</span> <span class="n">labels</span><span class="p">)</span><span class="o">.</span><span class="n">sum</span><span class="p">()</span><span class="o">.</span><span class="n">item</span><span class="p">()</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"> <span class="nb">print</span><span class="p">(</span><span class="s1">&#39;Test Accuracy of the model on the 10000 test images: </span><span class="si">{}</span><span class="s1"> %&#39;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="mi">100</span> <span class="o">*</span> <span class="n">correct</span> <span class="o">/</span> <span class="n">total</span><span class="p">))</span> </span></span></code></pre></div><h3 id="22-时间序列预测" class="relative group">2.2 时间序列预测 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#22-%e6%97%b6%e9%97%b4%e5%ba%8f%e5%88%97%e9%a2%84%e6%b5%8b" aria-label="Anchor">#</a></span></h3><p>Todo</p> <h3 id="23-图像输出文字" class="relative group">2.3 图像输出文字 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#23-%e5%9b%be%e5%83%8f%e8%be%93%e5%87%ba%e6%96%87%e5%ad%97" aria-label="Anchor">#</a></span></h3><p>Todo</p> <h2 id="补充" class="relative group">补充 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e8%a1%a5%e5%85%85" aria-label="Anchor">#</a></span></h2><ol> <li> <p>在原始发表文献用的图示是类似于下图的这种，看起来比较好容易理解当初形成LSTM的原因 <figure><img src="https://youngforever.tech/images/LSTM/LSTM_O.jpg" alt="LSTM_O" class="mx-auto my-0 rounded-md" /> </figure> </p> </li> <li> <p>pytorch lstm函数用法示例</p> </li> </ol> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="n">rnn</span> <span class="o">=</span> <span class="n">nn</span><span class="o">.</span><span class="n">LSTM</span><span class="p">(</span><span class="mi">10</span><span class="p">,</span> <span class="mi">20</span><span class="p">,</span> <span class="mi">2</span><span class="p">)</span> <span class="c1"># input_size, hidden_size, num_layers</span> </span></span><span class="line"><span class="cl"><span class="nb">input</span> <span class="o">=</span> <span class="n">torch</span><span class="o">.</span><span class="n">randn</span><span class="p">(</span><span class="mi">5</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">10</span><span class="p">)</span> <span class="c1"># time_step, batch, input_size（这里input_size即features）</span> </span></span><span class="line"><span class="cl"><span class="n">h0</span> <span class="o">=</span> <span class="n">torch</span><span class="o">.</span><span class="n">randn</span><span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">20</span><span class="p">)</span> <span class="c1"># num_layers, batch, hidden_size</span> </span></span><span class="line"><span class="cl"><span class="n">c0</span> <span class="o">=</span> <span class="n">torch</span><span class="o">.</span><span class="n">randn</span><span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">20</span><span class="p">)</span> <span class="c1"># num_layers, batch, hidden_size</span> </span></span><span class="line"><span class="cl"><span class="n">output</span><span class="p">,</span> <span class="p">(</span><span class="n">hn</span><span class="p">,</span> <span class="n">cn</span><span class="p">)</span> <span class="o">=</span> <span class="n">rnn</span><span class="p">(</span><span class="nb">input</span><span class="p">,</span> <span class="p">(</span><span class="n">h0</span><span class="p">,</span> <span class="n">c0</span><span class="p">))</span> <span class="c1"># output包含从最后一层lstm中输出的ht。shape: time_step, batch, hidden_size</span> </span></span></code></pre></div><ul> <li> <p><code>hidden_size</code> is the number of units of your LSTM cell. This means all the layers (input, forget, etc.) will have this size</p> </li> <li> <p>hidden_size即pytorch隐含层每个结构中含有的隐含cell数目</p> </li> </ul> <ol start="3"> <li>lstm函数中加入<code>bidirectional=True</code>参数即双向神经网络</li> </ol> <h2 id="reference" class="relative group">Reference <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#reference" aria-label="Anchor">#</a></span></h2><ol> <li>理解LSTM(<a href="http://colah.github.io/posts/2015-08-Understanding-LSTMs/" target="_blank" rel="noreferrer">http://colah.github.io/posts/2015-08-Understanding-LSTMs/</a>)</li> <li>高效RNN(<a href="http://karpathy.github.io/2015/05/21/rnn-effectiveness/" target="_blank" rel="noreferrer">http://karpathy.github.io/2015/05/21/rnn-effectiveness/</a>)</li> <li>Hochreiter &amp; Schmidhuber (1997) LSTM</li> <li>Pytorch LSTM官方文档(<a href="https://pytorch.org/docs/stable/nn.html#lstm" target="_blank" rel="noreferrer">https://pytorch.org/docs/stable/nn.html#lstm</a>)</li> </ol>

2020/3/9
阅读更多

PyTorch深度学习（1）

<script> document.addEventListener("DOMContentLoaded", function() { renderMathInElement(document.body, { delimiters: [ {left: '$$', right: '$$', display: true}, {left: '$', right: '$', display: false}, {left: '\\(', right: '\\)', display: false}, {left: '\\[', right: '\\]', display: true} ], throwOnError : false }); }); </script> <h2 id="1-history-motivation-and-evolution-of-deep-learning" class="relative group">1. History, motivation and evolution of Deep Learning <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#1-history-motivation-and-evolution-of-deep-learning" aria-label="Anchor">#</a></span></h2><p>科学技术发展如海浪一样也会潮起潮落，深度学习在经历了几次低谷后。2010年左右，在语音识别领域取得进展，2012年在计算机视觉领域也发展起来，随后各个领域都开始使用应用深度学习方法，而似乎渐渐抛弃了其他方法，那么深度学习是不是问题的最终解决之道呢？研究方向宽泛而多维才是合理的道路，不应过分追求热点领域。正如上世纪80年代日本学者在低谷时期仍然坚持自己的研究领域。</p> <p>学习表征：如何学习好的表征是深度学习要解决的问题之一，原始数据以一种有用的形式返回。自然状态下数据相互依赖有关系的。高效的表达方式应该是每类数据都是完全独立能完全单独表达某个方面。</p> <ul> <li>space tiling</li> <li>random projections</li> <li>polynomial classifier</li> <li>radial basis functions</li> <li>kernel machines</li> </ul> <h2 id="2-gradient-descent-and-backpropagation" class="relative group">2. Gradient Descent and Backpropagation <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#2-gradient-descent-and-backpropagation" aria-label="Anchor">#</a></span></h2><h3 id="21-gradient-descent" class="relative group">2.1 Gradient Descent <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#21-gradient-descent" aria-label="Anchor">#</a></span></h3><p>$$J(w, b) = \frac{1}{m}\sum_1^mL(\hat{y}^{(i)}, y^{i})$$ J(w, b)为问题的cost function即目标函数，即m个样本的损失函数平均值。使目标函数最小得到此时w,b参数是我们的优化问题。</p> <h4 id="211-梯度下降batch-gradient-descent" class="relative group">2.1.1 梯度下降(batch gradient descent) <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#211-%e6%a2%af%e5%ba%a6%e4%b8%8b%e9%99%8dbatch-gradient-descent" aria-label="Anchor">#</a></span></h4><p>梯度下降即上式对所有样本计算求出目标函数，通过对w,b求梯度来找到目标函数最小值，常用的一个比喻即找最快路径下山。数学理解是算法实现的重要一步，但与在计算机上实现还是有区别的，那么实际做法是什么样的呢？</p> <p>当你对复杂的问题想不清楚时，我们都可以从一个简单的例子出发来简化问题，对于这个问题考虑只有一个样本时，我们怎么编程实现呢？对w1、 b1，计算一个样本的loss然后对w1、b1求导优化思路很清晰，那么有m个样本的时候呢？只需将其他样本计算loss，然后对w1、b1求导相加。最后在通过学习率来更新w、b。可以看到每次更新都需要进行m次运算</p> <h4 id="212-小样本梯度下降mini-batch-gradient-descent" class="relative group">2.1.2 小样本梯度下降（mini-batch gradient descent） <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#212-%e5%b0%8f%e6%a0%b7%e6%9c%ac%e6%a2%af%e5%ba%a6%e4%b8%8b%e9%99%8dmini-batch-gradient-descent" aria-label="Anchor">#</a></span></h4><p>在每次更新时用n个样本，不用全部的样本。在深度学习中常用这种方法。用mini-batch可以享受向量化带来的便利，也不用全梯度下降那么大计算量，同时这也是应对冗余数据的一种方法。</p> <h4 id="213-随机梯度下降stochastic-gradient-descent" class="relative group">2.1.3 随机梯度下降(stochastic gradient descent) <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#213-%e9%9a%8f%e6%9c%ba%e6%a2%af%e5%ba%a6%e4%b8%8b%e9%99%8dstochastic-gradient-descent" aria-label="Anchor">#</a></span></h4><p>当n = 1的时候，每次更新的时候用1个样本。该方法在大多数情况下比全样本的梯度下降要快。</p> <p>三种优化方法最后收敛吗？最后能达到全局最小值吗？这是优化方法都需要考虑到的。可以阅读Optimization Methods for Large-Scale Machine Learning，我自己还没读过&hellip;</p> <h3 id="22-backprop" class="relative group">2.2 Backprop <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#22-backprop" aria-label="Anchor">#</a></span></h3><p>反向传播是为了求梯度用到的微积分链式法则，从而使梯度下降算法运行。</p> <h3 id="23-pytorch训练神经网络步骤" class="relative group">2.3 PyTorch训练神经网络步骤 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#23-pytorch%e8%ae%ad%e7%bb%83%e7%a5%9e%e7%bb%8f%e7%bd%91%e7%bb%9c%e6%ad%a5%e9%aa%a4" aria-label="Anchor">#</a></span></h3><ol> <li>output = model(input) 即神经网络前向传播</li> <li>J = loss(output, label) 计算cost function</li> <li>model.zero_grad() 清除梯度计算</li> <li>J.backward() 对requires_grad = True的变量计算梯度</li> <li>optimiser.step() 进行梯度下降</li> </ol> <h2 id="3-总结" class="relative group">3. 总结 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#3-%e6%80%bb%e7%bb%93" aria-label="Anchor">#</a></span></h2><p>看了前两节，觉得还是吴恩达大佬讲的好一些。建议网页上快速过内容即可，视频不用细看。</p>

2020/3/5
阅读更多

Mathematical notation

<p> <script> document.addEventListener("DOMContentLoaded", function() { renderMathInElement(document.body, { delimiters: [ {left: '$$', right: '$$', display: true}, {left: '$', right: '$', display: false}, {left: '\\(', right: '\\)', display: false}, {left: '\\[', right: '\\]', display: true} ], throwOnError : false }); }); </script> KaTeX can be used to render mathematical notation within articles.</p> <h2 id="inline-notation" class="relative group">Inline notation <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#inline-notation" aria-label="Anchor">#</a></span></h2><p>Inline notation can be generated by wrapping the expression in <code>\\(</code> and <code>\\)</code> delimiters.</p> <p><strong>Example:</strong></p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-tex" data-lang="tex"><span class="line"><span class="cl"><span class="c">% KaTeX inline notation </span></span></span><span class="line"><span class="cl"><span class="c"></span>Inline notation: <span class="k">\\</span>(<span class="k">\varphi</span> = <span class="k">\dfrac</span><span class="nb">{</span>1+<span class="k">\sqrt</span>5<span class="nb">}{</span>2<span class="nb">}</span>= 1.6180339887…<span class="k">\\</span>) </span></span></code></pre></div><p>Inline notation: \(\varphi = \dfrac{1+\sqrt5}{2}= 1.6180339887…\)</p> <h2 id="block-notation" class="relative group">Block notation <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#block-notation" aria-label="Anchor">#</a></span></h2><p>Alternatively, block notation can be generated using <code>$$</code> delimiters. This will output the expression in its own HTML block.</p> <p><strong>Example:</strong></p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-tex" data-lang="tex"><span class="line"><span class="cl"><span class="c">% KaTeX block notation </span></span></span><span class="line"><span class="cl"><span class="c"></span><span class="sb">$$</span><span class="nb"> </span></span></span><span class="line"><span class="cl"><span class="nb"> </span><span class="nv">\varphi</span><span class="nb"> </span><span class="o">=</span><span class="nb"> </span><span class="m">1</span><span class="o">+</span><span class="nv">\frac</span><span class="nb">{</span><span class="m">1</span><span class="nb">} {</span><span class="m">1</span><span class="o">+</span><span class="nv">\frac</span><span class="nb">{</span><span class="m">1</span><span class="nb">} {</span><span class="m">1</span><span class="o">+</span><span class="nv">\frac</span><span class="nb">{</span><span class="m">1</span><span class="nb">} {</span><span class="m">1</span><span class="o">+</span><span class="nv">\cdots</span><span class="nb">} } } </span></span></span><span class="line"><span class="cl"><span class="nb"></span><span class="s">$$</span> </span></span></code></pre></div><p>$$ \varphi = 1+\frac{1} {1+\frac{1} {1+\frac{1} {1+\cdots} } } $$</p>

2019/3/8
阅读更多

Git常用命令

<p>以下操作基于macOS，Windows仅供参考。</p> <h2 id="git初始化文件夹" class="relative group">git初始化文件夹 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#git%e5%88%9d%e5%a7%8b%e5%8c%96%e6%96%87%e4%bb%b6%e5%a4%b9" aria-label="Anchor">#</a></span></h2><p>进入目录</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">git init </span></span></code></pre></div><h2 id="新建gitignore" class="relative group">新建.gitignore <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#%e6%96%b0%e5%bb%bagitignore" aria-label="Anchor">#</a></span></h2><p>然后在其中加入需要忽略的文件或文件夹.gitignore 例如public\</p> <h2 id="git删除ds_store文件" class="relative group">git删除.DS_Store文件 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#git%e5%88%a0%e9%99%a4ds_store%e6%96%87%e4%bb%b6" aria-label="Anchor">#</a></span></h2><ol> <li>从该仓库中删除已存在的DS_Store文件<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup></li> </ol> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">find . -name .DS_Store -print0 <span class="p">|</span> xargs -0 git rm -f --ignore-unmatch </span></span></code></pre></div><ol start="2"> <li>新建<code>.gitignore_global</code>文件并将.DS_Store以及*/.DS_Store加入其中</li> </ol> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">vi .gitignore_global </span></span><span class="line"><span class="cl"><span class="c1"># 写入.DS_Store，*/.DS_Store</span> </span></span><span class="line"><span class="cl">git config --global core.excludesfile ~/.gitignore_global </span></span></code></pre></div><ol start="3"> <li>推到仓库</li> </ol> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">git add .gitignore </span></span><span class="line"><span class="cl">git commit -m <span class="s1">&#39;.DS_Store banished!&#39;</span> </span></span></code></pre></div><ol start="4"> <li>检查仓库中是否还有</li> </ol> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">git status </span></span></code></pre></div><h2 id="git删除远程分支文件" class="relative group">git删除远程分支文件 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#git%e5%88%a0%e9%99%a4%e8%bf%9c%e7%a8%8b%e5%88%86%e6%94%af%e6%96%87%e4%bb%b6" aria-label="Anchor">#</a></span></h2><p>当我们需要删除暂存区或分支上的文件, 同时工作区也不需要这个文件了, 可以使用</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">git rm file_path </span></span></code></pre></div><p>当我们需要删除暂存区或分支上的文件, 但本地又需要使用, 只是不希望这个文件被版本控制, 可以使用</p> <pre tabindex="0"><code>git rm –cached file_path </code></pre><p>所以我们经常使用以下命令来删除git中的文件</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">git rm -r --cached filename </span></span><span class="line"><span class="cl">git commit -m <span class="s1">&#39;delete some file&#39;</span> </span></span><span class="line"><span class="cl">git push origin master </span></span></code></pre></div><h2 id="git冲突处理" class="relative group">git冲突处理 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#git%e5%86%b2%e7%aa%81%e5%a4%84%e7%90%86" aria-label="Anchor">#</a></span></h2><p>git远程分支修改，本地也修改了准备提交出现冲突</p> <ul> <li>先拉在推 <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">git pull --rebase <span class="c1">#检查合并是否冲突</span> </span></span><span class="line"><span class="cl">git push -u origin master </span></span></code></pre></div></li> <li>强制按本地更新 <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">git push -f </span></span></code></pre></div></li> </ul> <h2 id="git子模块submodule" class="relative group">git子模块（submodule） <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#git%e5%ad%90%e6%a8%a1%e5%9d%97submodule" aria-label="Anchor">#</a></span></h2><p>对于公共资源或者常用的代码，你可能会把最新版本逐个复制到N个项目中，如果使用了submodule模块，那么只需要在各个项目中</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">git submodule update </span></span></code></pre></div><p>进入子模块目录正常操作即可</p> <h2 id="git多账户切换" class="relative group">git多账户切换 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#git%e5%a4%9a%e8%b4%a6%e6%88%b7%e5%88%87%e6%8d%a2" aria-label="Anchor">#</a></span></h2><p>删除keychain access种存储账户</p> <div class="footnotes" role="doc-endnotes"> <hr> <ol> <li id="fn:1"> <p><a href="https://www.jianshu.com/p/e3d8eb2a4295" target="_blank" rel="noreferrer"> Git删除.DS_Store</a>&#160;<a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> </ol> </div>

2019/3/4
阅读更多