• Real world graphs are essential to research
  • Great discoveries of the last 15 years all came from measuring real world graphs
  • Models were later built to capture these features
• The next great discoveries will also come from measured graphs
  • Graph dynamics
  • Interactions, influence, and contagion
  • Etc.
• How do researchers acquire real world graphs?
  • Not as straightforward as it seems
  • Many practical and theoretical challenges to collecting real graphs

• Go to a company and beg for data
• Go ask researchers for their data
• Go gather data yourself

• Advantages
  • Quick and easy; no need to spend time crawling
  • Possibility of getting complete data
  • Access to hidden metadata, e.g.
• Disadvantages
  • Most companies don't give data to unknown researchers
    • Privacy: user data is often sensitive
    • Economics: big data is often worth big money
  • Company may not give you what you want
    • Example: willing to give you the graph, but not user interactions
  • You may be forced to sign an NDA
    • May give the company right of refusal over your research findings
    • Prevents you from sharing data with other researchers

• Advantages
  • Quick and easy; somebody already did all the work for you
  • Basic analysis has probably already been done
• Disadvantages
  • Set of real graphs in academia is quite limited
    • Many graphs are incomplete, of suspicious quality
    • Many are tiny (order of thousands of nodes)
    • Almost all are simple, static snapshots: they lack all metadata and temporal information
  • Some researchers don't release their data
  • Some researchers can't release their data
    • Remember those NDAs?

• By far, the hardest method
  • But in many cases, the only method
• Practical challenges
  • If the site doesn't have an API, you have to scrape it
  • Gathering lots of data requires lots of time
    • Access to most sites is strictly rate limited
  • Data may be hidden by privacy settings
  • Gathering/storing data may violate website's Terms of Service
• Crawling methodology may introduce bias into the data
  • Graphs are dynamic, constantly evolving
  • What about disconnected components?
  • Traditional graphs traversal methods (e.g. BFS, DFS) are biased

• Photo sharing site
  • Users may follow each other
  • Graph is directed
• API for querying users and who they follow
  • In other words, only forward links can be crawled
• Crawling strategy
  • Pick seed users
  • Perform a BFS of the graph, starting at the seeds

• Use uniform random selection to locate an unbiased sample of users
  • In this case, 26.9% of 6902 random users where in the crawled data
• Are the remaining 73.1% of Flickr users "important"?
  • Only 250 random users (5%) were connected to the WCC
  • 89.7% of random users have zero forward links
  • Only 11K new nodes discovered by BFSing from the 5043 random users
• Thus, it is likely that the crawl captured the majority of the large WCC
  • Correlary: nodes disconnected from the large WCC only form small subgraphs

• Blogging site
  • Users may follow each others blogs
  • Following is much more common than on Flickr
  • Graph is directed
• API for querying users, who they follow, and who follows them
  • Forward links and reverse links can be crawled
  • Thus, possible to completely crawl the large WCC
• Completeness estimated via random sampling
  • 4773 random users out of 5000 (95.4%) present in the WCC
  • Thus, the crawl is highly likely to be comprehensive

• "Pure" social networking site
  • Users friend each other, both sides must approve
  • Graph is undirected
  • Users must be invited to join by a friend
  • Thus, Orkut forms a single, large SCC
• Challenges
  • No API, HTTP requests are rate-limited, user population is very large
  • Thus, a complete BFS crawl is impossible, even using 58 machines for >1 month
• Crawl gathered 3M of Orkuts 27M users (11.3%)

• Also known as Snowball sampling
• Partial BFS is biased towards gathering high degree nodes
• By definition the crawler is more likely to encounter high degree nodes
  • Example: there are only two paths to locating user with degree = 2
  • ... whereas there are 5000 paths to locate a user with degree = 5000
• Thus, the degree distribution of the Orkut sample is not representative
  • Other metrics are not as severely impacted by this bias
  • E.g. clustering and assortativity

• Video sharing site
  • Users may follow each other
  • Graph is directed
• API for querying users and who they follow
  • In other words, only forward links can be crawled
• Estimating Completeness
  • User identifiers are user-generated strings, not numbers
  • Thus, choosing a random sample of users is much more difficult (although not impossible)
  • No estimates of completeness in this paper

• Perform a complete BFS
  • How long will this take?
  • If the graph is directed, how many nodes are missing from the WCC?
• Gather a uniformly random sample of nodes
  • May not be possible on sites with irregular identifiers
• Partial BFS/DFS, a.k.a. Snowball Sampling
  • Known to produce results with biased degree distribution
• Random Walk
  • Also produces results biased towards high degree nodes

• Assuming complete BFS and uniform random sampling are impossible...
  • ... is there a way to sample the graph without incurring bias?
• Metropolis-Hastings random walk
  • MCMC technique for sampling from a difficult to sample distribution
  • Converges towards uniform random sampling

v = initial_seed_node
while stop_the_crawl is False:
  w = random.choice(v.neighbors)     # pick a neighbor at random
  p = random.random()                # 0 <= p < 1.0
  if p <= v.degree/w.degree: v = w   # walk to node w
  else: pass                         # stay at node v

• Not the only modified random walk algorithm for unbiased graph sampling
  • Whole subfield of research dedicated to these algorithms

• Twitter offers streams of tweets sampled uniformly at random (sort of)
• Spritzer: available for free, 1% of all tweets
• Larger hoses available from third-parties like Gnip for a fee
  • \($^5\) - Gardenhose, a.k.a. Decahose: 10% of all tweets
  • \($^6\) - Halfhose: 50% of all tweets
  • \($^7\) - Firehose: 100% of all tweets
• Twitter is (arguably) the most popular OSN for study because of this data

• Thinking critically about data gathering methodologies is key for evaluating study results
  • Is the data complete?
  • If the data is sampled, how was it sampled?
  • Are there hidden assumptions or biases in the data?
• Developing sound methodologies is crucial to conducting your own research
  • Think carefully about what data you need
  • Is it possible to gather this data? If so, what are the challenges?
  • Can you demonstrate that the data is free from bias, or the bias is quantifiable?
• GIGO
  • Garbage In - Garbage Out
  • If you start with poor data, the results are going to be even worse

• Real world data is always good, but it isn't without challenges
  • Hard to obtain, often incomplete or biased
  • Few datasets in existence
    • Are findings statistically significant if you only have 1 graph?
• Possible solution: use models to generate synthetic graphs
  • Infinite graph samples can produce high statistical confidence
  • No privacy issues, easy to share data

• How do you fit a model to a given real world graph?
  • Models have parameters that must be set
  • How do you choose parameter values such that model output is close to a given real world graph?
• Existing models are feature based
  • Preferential Attachment produces power-law degree scaling
  • Wattz-Strogatz produces tight-clustering and low path lengths
  • Forest Fire produces shrinking graph diameters
  • What about other features, or combinations of features?
• Existing models are not future-proof
  • How can a model capture features that haven't been identified yet?

• Simple but very powerful set of concepts
• First, extract a distribution from a given real graph
  • The \(d\)K-distribution acts like a fingerprint for the real graph
  • Single parameter, \(d\), controls amount of information captured by the distribution
  • As \(d\) increases, more information is extracted
• Second, a generator synthesizes a new graph based on a \(d\)K-distribution
  • Resulting graph is known as a \(d\)K-graph
  • Generator is stochastic; possible to generate many \(d\)K-graphs
  • ... however, \(d\)K-graphs are also statistically similar to the original graph
  • Similarity increases as \(d\) gets larger

• Like a puzzle where all pieces are identical
  • Very easy to put all the pieces together
  • But, many possible arrangements

• Like a puzzle where some pieces are identical
  • More types of pieces mean more constraints on valid constructions
  • Relatively harder to put together, still easy in an absolute sense
  • Less possible arrangments than before, but still significant freedom

• Like a puzzle where no pieces are identical
  • Only one possible construction

• 0K - How many graphs have \(n\) nodes and average degree \(k\)?
• 1K - How many graphs have avg. degree \(k\) and a specific degree distribution?
  • Recall the experiment where we randomly rewired the edges of graphs...
• 2K - How many graphs have avg. degree \(k\), specific degree distribution, and JDD?
• 3K - How many graphs have avg. degree \(k\), specific degree distribution, JDD, and clustering distribution?
• 4K - Etc...

• Method for deriving distributions from real graphs...
  • ... and generating synthetic graphs from those distributions
• Advantages over existing models
  • Based purely on graph structure, not tied to specific features
  • As \(d\) heads to infinity, captures target graph with perfect accuracy
  • In practice, \(d = 3\) is sufficient to capture all currently known features
  • No need to tune individual model parameters
• Disadvantages
  • Generators for \(d > 2\) are extremely slow and complex
  • Currently, there is no 3K generator capable of synthesizing large graphs