Blog
2022-02-17T20:19:34+00:00
http://craigchristenson.github.com
Craig Christenson
Decoding JSON to a Python Object
2013-05-16T00:00:00+00:00
http://craigchristenson.github.com/posts/python-json
<p>I recently had the opportunity to work on a Python project that involved pulling data from a JSON API. Python’s JSON module makes serializing and deserializing a breeze but for this project I also wanted to be able to call some custom methods on the dictionary returned after deserializing. The solution turned out to be pretty strait forward so I thought I would share what I came up with in case it is useful to anyone with a similar issue.</p>
<p>Create a new parent class which can get and set the attributes from the deserialized JSON dictionary.</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="k">class</span> <span class="nc">MyObject</span><span class="p">(</span><span class="nb">dict</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">dict_</span><span class="p">):</span>
<span class="nb">super</span><span class="p">(</span><span class="n">MyObject</span><span class="p">,</span> <span class="bp">self</span><span class="p">).</span><span class="n">__init__</span><span class="p">(</span><span class="n">dict_</span><span class="p">)</span>
<span class="k">for</span> <span class="n">key</span> <span class="ow">in</span> <span class="bp">self</span><span class="p">:</span>
<span class="n">item</span> <span class="o">=</span> <span class="bp">self</span><span class="p">[</span><span class="n">key</span><span class="p">]</span>
<span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">item</span><span class="p">,</span> <span class="nb">list</span><span class="p">):</span>
<span class="k">for</span> <span class="nb">id</span><span class="p">,</span> <span class="n">it</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">item</span><span class="p">):</span>
<span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">it</span><span class="p">,</span> <span class="nb">dict</span><span class="p">):</span>
<span class="n">item</span><span class="p">[</span><span class="nb">id</span><span class="p">]</span> <span class="o">=</span> <span class="bp">self</span><span class="p">.</span><span class="n">__class__</span><span class="p">(</span><span class="n">it</span><span class="p">)</span>
<span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">item</span><span class="p">,</span> <span class="nb">dict</span><span class="p">):</span>
<span class="bp">self</span><span class="p">[</span><span class="n">key</span><span class="p">]</span> <span class="o">=</span> <span class="bp">self</span><span class="p">.</span><span class="n">__class__</span><span class="p">(</span><span class="n">item</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">__getattr__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">key</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">self</span><span class="p">[</span><span class="n">key</span><span class="p">]</span></code></pre></figure>
<p>Then just create a subclass where you will call your custom methods for that object.</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="k">class</span> <span class="nc">Tweets</span><span class="p">(</span><span class="n">MyObject</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">dict_</span><span class="p">):</span>
<span class="nb">super</span><span class="p">(</span><span class="bp">self</span><span class="p">.</span><span class="n">__class__</span><span class="p">,</span> <span class="bp">self</span><span class="p">).</span><span class="n">__init__</span><span class="p">(</span><span class="n">dict_</span><span class="p">)</span></code></pre></figure>
<p>Now you can just initialize a new instance of the subclass with the deserialized JSON dictionary and all of your custom methods will be available.</p>
<h3 id="example-using-twitters-json-api">Example using Twitter’s JSON API</h3>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="kn">import</span> <span class="nn">urllib</span>
<span class="kn">import</span> <span class="nn">urllib2</span>
<span class="kn">import</span> <span class="nn">json</span>
<span class="k">class</span> <span class="nc">MyObject</span><span class="p">(</span><span class="nb">dict</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">dict_</span><span class="p">):</span>
<span class="nb">super</span><span class="p">(</span><span class="n">MyObject</span><span class="p">,</span> <span class="bp">self</span><span class="p">).</span><span class="n">__init__</span><span class="p">(</span><span class="n">dict_</span><span class="p">)</span>
<span class="k">for</span> <span class="n">key</span> <span class="ow">in</span> <span class="bp">self</span><span class="p">:</span>
<span class="n">item</span> <span class="o">=</span> <span class="bp">self</span><span class="p">[</span><span class="n">key</span><span class="p">]</span>
<span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">item</span><span class="p">,</span> <span class="nb">list</span><span class="p">):</span>
<span class="k">for</span> <span class="nb">id</span><span class="p">,</span> <span class="n">it</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">item</span><span class="p">):</span>
<span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">it</span><span class="p">,</span> <span class="nb">dict</span><span class="p">):</span>
<span class="n">item</span><span class="p">[</span><span class="nb">id</span><span class="p">]</span> <span class="o">=</span> <span class="bp">self</span><span class="p">.</span><span class="n">__class__</span><span class="p">(</span><span class="n">it</span><span class="p">)</span>
<span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">item</span><span class="p">,</span> <span class="nb">dict</span><span class="p">):</span>
<span class="bp">self</span><span class="p">[</span><span class="n">key</span><span class="p">]</span> <span class="o">=</span> <span class="bp">self</span><span class="p">.</span><span class="n">__class__</span><span class="p">(</span><span class="n">item</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">__getattr__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">key</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">self</span><span class="p">[</span><span class="n">key</span><span class="p">]</span>
<span class="k">class</span> <span class="nc">Tweets</span><span class="p">(</span><span class="n">MyObject</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">dict_</span><span class="p">):</span>
<span class="nb">super</span><span class="p">(</span><span class="bp">self</span><span class="p">.</span><span class="n">__class__</span><span class="p">,</span> <span class="bp">self</span><span class="p">).</span><span class="n">__init__</span><span class="p">(</span><span class="n">dict_</span><span class="p">)</span>
<span class="o">@</span><span class="nb">classmethod</span>
<span class="k">def</span> <span class="nf">search</span><span class="p">(</span><span class="n">cls</span><span class="p">):</span>
<span class="n">req</span> <span class="o">=</span> <span class="n">urllib2</span><span class="p">.</span><span class="n">Request</span><span class="p">(</span><span class="s">'http://search.twitter.com/search.json?q=%40twitterapi'</span><span class="p">)</span>
<span class="n">result</span> <span class="o">=</span> <span class="n">urllib2</span><span class="p">.</span><span class="n">urlopen</span><span class="p">(</span><span class="n">req</span><span class="p">).</span><span class="n">read</span><span class="p">()</span>
<span class="k">return</span> <span class="n">Tweets</span><span class="p">(</span><span class="n">json</span><span class="p">.</span><span class="n">loads</span><span class="p">(</span><span class="n">result</span><span class="p">))</span>
<span class="k">def</span> <span class="nf">get_tweets</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">tweet_data</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">()</span>
<span class="k">for</span> <span class="n">tweet</span> <span class="ow">in</span> <span class="bp">self</span><span class="p">.</span><span class="n">results</span><span class="p">:</span>
<span class="n">tweet_data</span><span class="p">[</span><span class="n">tweet</span><span class="p">.</span><span class="n">created_at</span><span class="p">]</span> <span class="o">=</span> <span class="n">tweet</span><span class="p">.</span><span class="n">text</span>
<span class="k">return</span> <span class="n">tweet_data</span>
<span class="c1"># get all tweet data
</span><span class="n">query</span> <span class="o">=</span> <span class="n">Tweets</span><span class="p">.</span><span class="n">search</span><span class="p">()</span>
<span class="c1"># create a new dictionary using the tweet time and text
</span><span class="n">tweets</span> <span class="o">=</span> <span class="n">query</span><span class="p">.</span><span class="n">get_tweets</span><span class="p">()</span>
<span class="c1"># pretty print
</span><span class="k">print</span><span class="p">(</span><span class="n">json</span><span class="p">.</span><span class="n">dumps</span><span class="p">(</span><span class="n">tweets</span><span class="p">,</span> <span class="n">indent</span><span class="o">=</span><span class="mi">4</span><span class="p">))</span></code></pre></figure>
<h3 id="result">Result</h3>
<figure class="highlight"><pre><code class="language-json" data-lang="json"><span class="p">{</span><span class="w">
</span><span class="nl">"Thu, 16 May 2013 21:14:02 +0000"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Download GrassHeads - Once In A LifeTime ft. Gill.mp3 | Limelinx http://t.co/gQ91Lib6EN via @twitterapi"</span><span class="p">,</span><span class="w">
</span><span class="nl">"Thu, 16 May 2013 21:17:14 +0000"</span><span class="p">:</span><span class="w"> </span><span class="s2">"RT @SaskiaWaterland: Een bloemlezing van VPRO's seksuele hoogtepunten: de vpro.nl seksavond - VPRO http://t.co/aavuKweaAn via @twitterapi"</span><span class="p">,</span><span class="w">
</span><span class="nl">"Thu, 16 May 2013 21:14:44 +0000"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Download Tom Arts - Dont Blame Me.mp3 | Limelinx http://t.co/CeQIl9PW0W via @twitterapi"</span><span class="p">,</span><span class="w">
</span><span class="nl">"Thu, 16 May 2013 21:15:33 +0000"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Whatcom Locavore: Women becoming a major force in sustainable agriculture http://t.co/FGB8psGTGJ via @twitterapi"</span><span class="p">,</span><span class="w">
</span><span class="nl">"Thu, 16 May 2013 21:09:02 +0000"</span><span class="p">:</span><span class="w"> </span><span class="s2">"YANGON, Myanmar: Myanmar says Shan rebels attack oil company office | World | http://t.co/C9KZPeHEQF http://t.co/y4HFNRaMSt via @twitterapi"</span><span class="p">,</span><span class="w">
</span><span class="nl">"Thu, 16 May 2013 21:16:52 +0000"</span><span class="p">:</span><span class="w"> </span><span class="s2">"RALEIGH: Raleigh police get homicide victim</span><span class="se">\u</span><span class="s2">2019s deleted Facebook page in seeking leads | Crime | http://t.co/pIJZsxrAPs via @twitterapi"</span><span class="p">,</span><span class="w">
</span><span class="nl">"Thu, 16 May 2013 21:15:23 +0000"</span><span class="p">:</span><span class="w"> </span><span class="s2">"RT @TheOnlyPilot: Download Nick - Chopped and Screwed.mp3 | Limelinx http://t.co/XTK87Hy7cL via @twitterapi"</span><span class="p">,</span><span class="w">
</span><span class="nl">"Thu, 16 May 2013 21:13:10 +0000"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Kad ortak odvali nesto, a smeh jaci od tebe | Smesne reakcije @VladoGeorgiev http://t.co/h7bQHrKhmz via @twitterapi"</span><span class="p">,</span><span class="w">
</span><span class="nl">"Thu, 16 May 2013 21:09:04 +0000"</span><span class="p">:</span><span class="w"> </span><span class="s2">":)) Adam Lambert's fans land him an O Awards nomination: Pressparty http://t.co/dEjjs28gWa via @twitterapi"</span><span class="p">,</span><span class="w">
</span><span class="nl">"Thu, 16 May 2013 21:18:30 +0000"</span><span class="p">:</span><span class="w"> </span><span class="s2">"</span><span class="se">\u</span><span class="s2">00bb Rene Rancourt Visits Barstool HQ</span><span class="se">\u</span><span class="s2">2019s To Get Us Ready for B</span><span class="se">\u</span><span class="s2">2019s Vs. Rangers Barstool Sports: Boston via @twitterapi"</span><span class="p">,</span><span class="w">
</span><span class="nl">"Thu, 16 May 2013 21:09:28 +0000"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Download Nick - Chopped and Screwed.mp3 | Limelinx http://t.co/XTK87Hy7cL via @twitterapi"</span><span class="p">,</span><span class="w">
</span><span class="nl">"Thu, 16 May 2013 21:16:27 +0000"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Yaa check m</span><span class="se">\u</span><span class="s2">011b Out Go Download Transformer- Akrshome Foe.mp3 | Limelinx http://t.co/zvvWy20HuG via @twitterapi"</span><span class="p">,</span><span class="w">
</span><span class="nl">"Thu, 16 May 2013 21:13:57 +0000"</span><span class="p">:</span><span class="w"> </span><span class="s2">"(1) Valores Habbid - F</span><span class="se">\u</span><span class="s2">00f3rum </span><span class="se">\u</span><span class="s2">00b7 Habbid http://t.co/JL02isdniq via @twitterapi"</span><span class="p">,</span><span class="w">
</span><span class="nl">"Thu, 16 May 2013 21:14:17 +0000"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Washington, D.C.: B Reactor park bill passes Senate committee | Hanford news | http://t.co/9Kcvgf3PrY http://t.co/1SDhZPLT8A via @twitterapi"</span><span class="p">,</span><span class="w">
</span><span class="nl">"Thu, 16 May 2013 21:10:13 +0000"</span><span class="p">:</span><span class="w"> </span><span class="s2">"RT @GlamPimpLinda: Adam Lambert's fans land him an O Awards nomination: Pressparty http://t.co/xgJODYkYPK via @twitterapi"</span><span class="w">
</span><span class="p">}</span></code></pre></figure>
<p>Hopefully this little tutorial helps if you run into the same issue.</p>