Where's the IDE (Integrated Development Environment) for personal information?

I write down everything I want to remember. That way, instead of spending a lot of time trying to remember what it is I wrote down, I spend the time looking for the paper I wrote it down on. -- Beryl Pfizer

I've dreamed up an information-managing tool that should exist, and it's driving me nuts working without it. Here are a few initial notes; comments are very welcome.

In training myself to be a productivity consultant, my information tracking and capturing needs have gone through the roof. For example, I need to manage things like:

• networking information (who, when, how I connected to them),
  
• blog ideas,
  
• useful productivity tips for myself and clients,
  
• marketing and web site ideas,
  
• notes from books I read [2],
  
• notes/ideas for projects,
  
• lessons learned and personal challenges,
  
• inspirational reading,
  
• contact information,
  
• project hours (for example, in a coaching exchange I'm doing with someone),
  
• ...

(Note that I don't use this file for my GTD implementation; for that I use a paper planner with sections for my calendar, Next Actions, projects, etc. [3])

As I've described before [4], I use a plain text file with simple formatting to break it into individual records, and WikiCase (AKA CamelCase or WikiWords) to help navigate and find needed information. Here's an excerpt:

  ----
  IDEA: IdeaMatt: from JasonWomack: you teach what you most need to
  learn. in my case: ...
  (2005-11-15 21:40:42)
  ----
  IDEA: IdeaMatt: the irony of trying to set something up (take action)
  with people who need help taking action - slow email responses,
  unreturned phone calls, etc.
  (2005-11-15 14:38:14)
  ----
  IDEA: IdeaMatt: idea of copying todo lists when reach 1/2 full
  from "Time management for dummies"
  (2005-11-15 13:40:49)
  ----
  ...
  ----

After using this scheme for over a year, I've realized the tool is ready to grow up, and what it wants to be is something akin to a programmer's tool for information. Now I'm the first to admit that I'm an information "power user," and that a tool that works well for me would probably make most people run screaming with tears streaming down their faces, but hey - my readers are smarties!

What I've in mind is something like the next level past Wikis (think Jot meets Dabble DB outside the mad doctor's castle, and gets attached by half-crazed Lotus Agenda townspeople carrying pitchforks and torches, runs smack into Chandler, and gets stabbed by Planner Modefor good measure). It would work like an information IDE, something like the stellar IntelliJ IDEA, but for semi-structured data.

Here are some of the features, with programming comparisons to IDEA:

• Views: The UI would have multiple views, including a hierarchical one like wikidPad, a graphical one like MindManager, and a chronological one like Lifestreams (the one my text file tries to emulate).
  
• Editing: Smart completion of WikiWords, time-stamping of entries, etc. NB: Completion is crucial for efficiency, and for consistent use of tags.
  
• Navigation: Basic link following, including Find Callers and Jump To Definition, bookmarks, and forward/back functionality. (Note: the graph view may be better for higher-level navigation and abstraction.)
  
• Reports: One thing I'd love to have is a histogram showing who I've talked to (WikiWords, more generally) on a timeline so that I can see who I've not had contact with in a while, what kinds of entries I've been focusing on, etc.
  
• Refactoring: Often I'll find that I've duplicated some concept (such as a blog idea or a project), and I'll want to do the equivalent of XP's refactoring, including Extract Entry, Create Link, Inline entry, Rename Entry, etc.

That's the initial thought. However, there's something deeper that goes beyond these relatively straightforward operations. On top of them, I see the need for what I've been calling "self-structuring" operations, ones that support emergent organization of the data, esp. for records with more traditional database-like structure [5]. More on this later, but it would move the idea from a tool to a framework for information applications, enabling it to specialize itself (with guidance) to tasks like finances, expenses, contacts, and what I call "semantic bookmarks."


Related Reading

For additional thoughts on this, interested readers might enjoy the following drafts:

• CRAM - a Contextual Relational Augmented Memory for Personal Data Mining
  
• An intelligent email system
  
• Ideas on a next-generation personal information manager

Note that they refer to our research project Proximity. It's a system supporting relational knowledge discovery (advanced data mining), but my original interest was as a framework for implementing some of the above ideas.


References

• [2] You might enjoy How to read a lot of books in a short time.
  
• [3] My current GTD setup is described in Fare thee well Hipster PDA - I barely knew ye.
  
• [4] You can read more about the text file format in My Big-Arse Text File - a Poor Man's Wiki+Blog+PIM, Pickle jars, text files, and creative idea capture, and Photo Blogs, Wikis, and Memories for Life.
  
• [5] My friend David Phillips also talks about this in Gems in a treasure chest.

How to read a lot of books in a short time

We now accept the fact that learning is a lifelong process of keeping abreast of change. And the most pressing task is to teach people how to learn. -- Peter F. Drucker

As part of my self-planned Master's Degree in Personal Productivity I've been reading non-fiction rather voraciously [1]. For example, I usually have 3-5 books going per week, with replacements arriving regularly [2]. The topics cover productivity, personal growth, consulting, networking, and creativity. However, a significant problem I've encountered is a certain "tyranny" of reading for education (rather than for entertainment), and it has threatened to turn the process into a form of aversion therapy.

Naturally, because my goal is to learn, the reading involves work. But the question is: How can one read efficiently, capture relevant ideas in a usable way, and keep the process sustainable and enjoyable? The rest of my post summarizes the best solutions I've found, but the most useful technique comes from Jason Womack [3], and synthesizes nicely the most common ideas. In a nutshell, he says he reads the book four times:

1. Table of contents, glossary, index.
2. Anything in bold, titles, and subtitles.
3. First line of every paragraph.
4. Entire book

Here's the twist: Steps 1-3 should only take about 10 minutes. To capture relevant information he uses a note-taking scheme involving putting dots in margins, and cross-referencing them in an index in the book's front. When done, he transfers them to a text file.

After adopting his system with a slight variation (I dictate my notes into an inexpensive cassette tape recorder, then transcribe them into my system [4]), I've found it works great. I can very quickly scan a book, decide if it's worth reading in depth (steps 3 and 4), and which sections are likely to be most relevant to my goals. My only other point is to note that I seem to need a balance between non-fiction and fiction. (My current ratio of non-fiction to fiction is about 5:1, but should probably be more like 3:1.)

I'd love to hear your suggestions and tricks!


Related methods

Following are related articles, each with its own twist. The common point, though, is to efficiently find ideas that are relevant to your goals, usually via some sort of skimming. The big change for me (some slight embarrassment here) was the realization that I didn't have to read the entire book word-for-word!

In How to read a business book, Brendon Connelly suggests marking up (tagging) interesting passages with a master index at the back of the book. He also contributes tips on where to read, pens to use, etc.

The classic How to Get Control of Your Time and Your Life (by Alan Lakein), suggests reading books like newspapers. The main points:

• Put a new book into your inbox every day, and take the old one out, even if unread.
• Then read the 'headlines' on jacket (most significant points).
• Then glance through it quickly, noting items of interest, taking about the same amount of time as reading a newspaper.
• The goal: Find the key ideas and understand their applicability to your situation.
• Use the preface, table of contents, and summaries at beginning and end of book.
• Read details only if a) it's meaningful, and b) it's involved.

This approach allows gaining value in a surprisingly short time. It's good because you see more books, and are more likely to see really good ones. Also, you get efficient at skipping lower quality works.

In Open Loops: A Quick and Dirty Reading Strategy When Time is Short, the section "How To Find the Essential 20%" lists these points:

1. Read the title of the material.
2. Read the introduction.
3. Read the Table of Contents.
4. Flip through the material, scanning the chapter titles and sub-headings.
5. Look at the illustrations and captions.  Look at the charts and diagrams.  Read the pull-quotes and sidebars.
6. Scan through the index looking for your particular business’ buzz words.
7. Now read the first chapter (or in a shorter work, the first paragraph).
8. Flip through the book and read the first sentence of each paragraph.
9. Read the last chapter (or paragraph in a shorter work).  If there is an executive summary, read it.
10. Read any other information on the cover or dust jacket.

In How to Study and Make the Most of Your Time, an approach is presented that I found was commonly recommended to students:

• Schedule important work.
• Ask yourself questions as you read - read to answer questions.
• Use SQ3R: Survey, Question, Read, Recite, Review.
• Try to develop an overall concept of what you have read in your own words and thoughts. try to connect things you have just read to things you already know.
• Every paragraph contains a main idea - make it a habit to find the main idea in each paragraph you read.
• Think!

The article Leading Forward: How to Read and Digest a Book! (apparently gone, but still in - Google's cache) recommended five steps: Selection, Preparation, Read Actively, Reflect for Insight, Systemise for implementation. This was in the minority in that it addressed how to use the information after reading.

Finally, from The Great Big Book of Personal Productivity, by Ron Fry: To summarize the skimming process:

1. Read and be sure you understand the title or heading. Try rephrasing it as a question for further clarification of what you will read.
2. Examine all the subheadings, illustrations, and graphics. these will help you identify the significant matter within the text.
3. Read thoroughly the introductory paragraphs, the summary, and any questions at chapter's end.
4. Read the first sentence of every paragraph.this generally includes the main idea.
5. Evaluate what you have gained from this process: Can you answer the questions at the end of the chapter? Can you intelligently participate in a class discussion of the material?
6. Write a brief summary that capsulizes what you have learned from your skimming.
7. Based on this evaluation, decide whether a more thorough reading is required.

References

• [1] In his article Read a Book a Week, Steve Pavlina explains a meta reason for reading voraciously:

  But the actual knowledge and the new distinctions you gain from reading are not the main benefit. My experience has shown me that the real benefit comes not from what you read but rather from the habit of reading. When you read a new book every week, you condition your mind to keep taking in new knowledge. Your thinking remains fresh and sharp. Your brain is always churning on new ideas, looking for new distinctions it can make. Every day you pour in more ideas, which your brain must find a way to integrate into your existing knowledge base. Frequent reading fires up your neural activity, even during the periods when you aren't reading.

• [2] Preferred sources: 1) My local library (which supports web-based requests), 2) Amazon's used marketplace, and 3) Amazon's new books. However, I've recently been exploring ebay's books section. I'd love to hear others' experiences buying from ebay...
• [3] Via personal correspondence.
• [4] See My Big-Arse Text File - a Poor Man's Wiki+Blog+PIM and Pickle jars, text files, and creative idea capture.

Why every problem should be a GTD project

David Allen is quoted many places [1] as saying in his GTD | The RoadMap seminar:

Anything that's a "problem" is a "project."

When I first heard this in Boston in late September my thought was "sure - makes sense" and that was about it. However, like much of what's behind GTD, it's simple but deep. In this case, I've found that creating a "full" GTD project for every single problem is a great idea, even if the problem seems to require just one step to solve, because the problem often becomes more complex, presents surprises, or needs to be tracked (i.e., doesn't go away the first time).

Here's a typical example (this seems to come up with billing errors in particular): I was recently charged for an optical exam that should have been covered by my insurance company [2]. Here are the steps:

1. Bill goes into paper collection point [3].
   
2. Process - What is it?
   
3. A bill for something that should have been covered. Ugh!
   
4. Add to @Calls: "Health Services re: glasses bill - 545-0111" [4]
   
5. Later, when time: Make the call, talk to (very helpful) person - promises all straightened out. Great!
   
6. Toss bill in recycling bin.

Oops.

Here's the problem: I naively assumed the problem was really taken care of. But guess what? Three weeks later, return to step 1. However, I don't have the original bill, I don't have the person's name I talked to, and I don't have a file that should contain all of this. So now, because I didn't do the basics, my stress level is higher than it should have been!

So now I've learned my lesson: I now use any problem as a trigger to create a project - I make an entry in my project list [5], create a folder with my labeler [6], and file any related paper work in it. Just. In. Case.

One question that comes up is: Doesn't this approach create a bunch of needless files? Well, I think it's better to have the files around in case I need them, than the alternative. I've created plenty of space in my office for filing (as Allen recommends) [7], so it's not a problem. And removing them in the yearly purge (it's on my checklist) is not a problem.

So is it a problem? Make it a project!


References

• [1] See, for example: buzznovation: GTD...the Road Map...San Jose and From the Belly of the Beasts: Getting Things Done: the Roadmap.
  
• [2] Note that I realize actually having insurance puts me in a certain class, which does not make me happy (see this Center on Budget and Policy Priorities story if you're interested). In other words, having an insurance problem is a privilege.
  
• [3] There's a photo of my office setup here, for the morbidly curious
  
• [4] I always try to write the phone number on my @Calls list. Plus, I stopped writing "Call ..." because they're all calls...
  
• [5] I'm now using Post-It notes for GTD projects, instead of lists.
  
• [6] Please, use the LetraTAG QX50 instead of Brother models, which waste a ton of tape. Also, the tape is of the expensive laminated variety. The Dymo, on the other hand, uses economical paper tape, doesn't waste, and runs forever on its batteries. I bring this up because I had a client who bought a Brother and was very upset about it. I now carry some extras with me just in case they're not available locally. (Note: You can sometimes get them from TigerDirect for about $10.)
  
• [7] Instead of buying cheapo filing cabinets from the big box office supply store, we go to an awesome local used office furniture store, which has a ton of inexpensive, but very high quality, filing cabinets. Our two-drawer cabinets are usually $300-400, but we got them for ~$50 each. They really work smoothly, and have have plenty of capacity. Plus, you never know what you'll find in them!
  
• Ramit's story I Will Teach You To Be Rich: Sprint tries to rip Ramit off? Oh my provides another example of why it's important to keep good problem records.
  
• In my article Some GTD warning signs I mention problems not being projects as a warning sign.
  
• For a great resource on problem-solving techniques, check out this MindTools article. (It's a great site overall, I've found.)
  
• In her book Free to Succeed: Designing the Life You Want in the New Free Agent Economy Barbara Reinhold encourages us to "Find the opportunity in the problem." In my case, I tried to get to know my eye care provider's office manager a little better, part of my effort to connect with three new people a week. (More in my article A geek "gets" networking: The strange magic of connecting with others.)
  
• The article Five Whys describes the motivation for asking "Why?" at least five times:

  When you are faced with a problem it is useful to stop and ask why five times. ... The aim is to ensure that a problem stays solved and never happens again. We do this by identifying and eliminating the root cause of the problem. When the root cause is eliminated the original problem is solved permanently. If the root cause is not dealt with the problem can happen again and again in the future.

  
  
• In his book How to Stop Worrying and Start Living, Dale Carnegie identifies some basic techniques in analyzing worry, which apply well to problem solving:
  1. Get all the facts.
     
  2. Weigh all the facts - then come to a decision.
     
  3. Once a decision is reached, act!
     
  4. Write out and answer the following questions:
     1. What is the problem?
        
     2. What are the causes of the problem?
        
     3. What are the possible solutions?
        
     4. What is the best possible solution?

Another GTD Plus - Moving offices made much easier

This week my wife and I decided to move our home office downstairs, primarily for increased convenience - easier access to in-baskets, chances for opportunistic workflow processing, closer connection to our daughter's activities, etc. Interestingly, we've both adopted David Allen's Getting Things Done methodology, and we were really surprised at how relatively easy this move was compared to others we've done.

Why? Well, the basics of any move consists of the physical mechanics of moving things to the new space, including disassembling desks, unloading filing cabinets, packing books, packing up the computers, etc. That part, while tiring, is moderately brainless. However, without a system like GTD, there's usually a significant amount of mental stress involved as well, especially around "stuff" (i.e., artifacts with associated un-made decisions). In other words, in addition to the exhausting physical part of the move, we also had to do some collecting, processing, deleting, filing, etc. (see Allen's workflow diagram - or Doug Johnston's variation). Instead, with our systems up and running smoothly, there are literally no loose ends - everything was in our systems, and our collection points were empty, which enabled us to focus on the purely mechanical process, without the added psychological weight and resistance.

This is related to the point David Allen makes in his seminars about the relief people feel when preparing for vacation - a sense of control that comes from being on top of everything (i.e., having a complete inventory, and no "pending" files or in-basket items). As I explained in Is GTD the "Extreme Programming" of Time Management?, adopting GTD means you get that sense of relief at least once per week (in the weekly review), and often daily.

(For the curious, I've uploaded and annotated a picture of my desk in the new office on flickr.)


Related threads from the davidco forums

• Moving Offices: Adopting GTD before a move has these rewards:
  1. a list of reminders rather than a pile of them,
     
  2. phone numbers entered rather than sitting in a pile or a file,
     
  3. most likely a reduction of total mass to move because you may encounter duplicates and get rid of the extras, and
     
  4. as you work you will think of projects that you want to embark on after the office move and if you have kept a record of them and how to proceed on them you may find that you are part-way to completion before you have even changed spaces
  
  
• big move coming up:

  Moving is a great opportunity to pare down and organize. But if you run short of time the temptation will be to just pack up all the junk as is and stuff it wherever it will fit in the new location. So start paring down ASAP.

• Please suggest the first action for this project!: Talks about GTD's advantages for processing the new space, if it's full of someone else's stuff.

Sidebar: Extreme Programming & GTD: Releases

This topic (moving) made me think of an XP similarity to GTD - releases. In traditional software development models (the waterfall), creating a release (e.g., version 4.0) is a big deal, involving lots of preparation, testing, getting together documentation, etc. However, following the Extreme Programming model makes it possible to release basically whenever it's convenient. The team is always working to keep the software ready at any point - helped by nightly builds and tests, along with the other XP practices.

The analog to GTD is that one is always ready for whatever's coming your way: Move the office tomorrow? Great! (Nothing much to do except do the physical move - you have very few loose ends.) Major project (read opportunity) dropped in your lap this morning? Fabulous! (You're ready to re-evaluate how this changes your work world because you know everything you've committed to.) As Jason Womack puts it - in life shift happens, and it's our responses to change that determine how gracefully we meet life's challenges. I've found Allen's work helps me be more agile, (and more relaxed), which helps me ultimately be more in tune with life' surprises.

Some GTD warning signs

As I've continued my practice of David Allen's Getting Things Done (GTD) methodology (and helped others adopt it) I've occasionally noticed some signs of falling behind. I wanted to share a few of them, and ask what others you've come across in your implementation [1]. In no particular order:

• "Cobwebby" mind (needs a sweeping).
  
• "Doing" while processing (unless 2 minute or less) - beware rabbit holes.
  
• "To file" or "Pending" trays.
  
• A lonely labeler.
  
• Calendar entries that don't absolutely have to be done that day.
  
• Collection points getting too big or old.
  
• Haven't done a weekly review in "a while."
  
• Keeping anything in your head (i.e., notice if something's bugging you).
  
• Next Actions that are out of date, too big, vague, not getting done, or that repel (check out Does this "next action" belong someplace else?).
  
• Not following the Processing & Organizing Diagram for your "stuff."
  
• Not using system daily, or not trusting it.
  
• Orphan papers (loose, with no home), or piles.
  
• Packed or unattractive file drawers (e.g., messy labels or old folders).
  
• Project list out of date - projects not listed, including problems (Allen says "anything that's a problem is a project").
  
• Putting off decisions, instead of making them when stuff enters your life.
  
• Relying on any of the following as reminders: project support folders, sticky notes, printouts, or loose notes. (Only use Calendar and Next Action lists.)
  
• Remembering to do something without looking at your Next Actions lists (e.g., recalling who to call when not looking at @calls).
  
• The existence of stealth projects - projects masquerading as Next Actions.

So what do you think - what warning signs have you seen?

P.S. A hearty congratulations to David Allen for "Getting Things Done" making it to the top of BusinessWeek's Best-Seller List - via Eric Mack. Thanks very much for your contribution!


References

[1] Readers interested in a positive summary might enjoy Mark Wieczorek's Ready To Test For Your Belt?