Blocks and queries

[dannbass]

Hi!
I didn't know if I'm posting in the right place, but here it goes...
I noticed that after I put the recent topics in the smf style my queries went up from 40 in  0.206 seconds to 73 in the same time, also I have the calendar with events and birthdays and is giving me 5 more queries... if I disable the left panel the board is 35 queries, with the left panel goes up to 40, and with the center panel jumps to 73 to 77.

The only java thing I have is the clock...
Any ideas?

My forum is by registration, so if you want to see it you'll need to use as username demo and  password demo.

Thanks!

[G6Cad]

It's the way both SMF and TP works.
The more content you have showing that uses any database calls, the longer time and also the more queries is required.
Recent topics (if set to standard 5 posts) you will have 5 more queries right there as it have to search for 5 more items to show.

[dannbass]

Thanks for the reply!
But then why is it 33 more queries just for the recent topics? I have to display 10 new topics...

How does it hurt my site? Am I in trouble or something like that?
Thanks G6!

[jacortina]

Which version of the recent topics are you using?

What happens to the query count if you refresh the page with recent topics on it?

[dannbass]

It is the same.

I'm using the latest version of the recent topics by you, but I removed the board names and move the author of the topic, so it will be more like my theme.

Here is the code, also I noticed that in the default the recent topics are 20 and in my theme only 10... (I just solved this!!)

Also, I thought that maybe since the theme is a custom one made in part by me, that I could have made tons of mistakes, so I installed the Pn-Pn version with TP and the count went up to 472 queries!!!!!!

Code Select Expand

global $context, $settings, $scripturl, $txt, $db_prefix, $ID_MEMBER, $user_info, $modSettings, $user_profile;

//////////////////////////////////////////// ---------- Unconditional Exclude
//
$exclude_boards = array(84, 85, 86, 87, 88, 89); // KEEP (to preserve variable declaration)
// $exclude_boards = array(5); //  Exclude single board
// $exclude_boards = array(5, 8); //  Exclude multiple boards
$ex_board_clause = !empty($exclude_boards) ? ' AND b.ID_BOARD NOT IN (' . implode(', ', $exclude_boards) . ')' : '';
//
//

//////////////////////////////////////////// ---------- Use in TP PHP Article (no title or frame from theme)
// This will give most recent XX posted to topics -OR-
// most recent XX unreplied to topics -OR-
// topics posted to in last XX hours -OR-
// most recent XX topics on topic notify list
//
// Sorts by most recent reply (descending; most recent first) -OR-
//          creation order (descending; most recent first)
//
// All in the detailed topic format
//
//
// Default
// index.php?page=##
// most recent posted to topics - will list
// number equal to  $settings['number_recent_posts']
//
// index.php?page=##;count=50 or index.php?page=##;type=last;count=50
// 50 most recently posted to topics
//
// index.php?page=##;type=unreplied or index.php?page=##;type=unreplied;count=50
// Most recent unreplied to topics - will
// list number specified in 'count' or default to
// number equal to  $settings['number_recent_posts']
//
// index.php?page=##;type=hours or index.php?page=##;type=hours;count=12
// Topics posted to in last number of hours
// specified in 'count' or default to 24.
//
// index.php?page=##;type=notify or index.php?page=##;type=notify;count=50
// Topics in topic notifcation list up to number
// specified in 'count' or a maximum of 100 topics.
//
// *** Admins Only ***
// index.php?page=##;type=notify;user=XXX or index.php?page=##;type=notify;user=XXX;count=50
// Topics in topic notifcation list of specified user up to number
// specified in 'count' or a maximum of 100 topics.
//
// Add argument order=create to sort by topic creation sequence (most recent first)
// rather than by last reply sequence

$do_query = 1;

//////////////////////////////////////////// ---------- Poor Man's Global Announcements Block (center block - no Title/Frame)
// Delete documentation comments above and marked section below
//
// $announce_topics = array(254, 568, 675, 678); // Topic ID's to be 'Announced'
//
// $heading = '<center>Announcements<center>';
// $where_clause = 't.ID_TOPIC IN (' . implode(', ', $announce_topics) . ')';
// $limit_clause = '';
// $order_clause = 't.ID_LAST_MSG DESC';
////////////////////////////////////////////   

//////////////////////////////////////////// ---------- Last 5 Topics Started by User Block (center block - no Title/Frame)
// Delete documentation comments above and marked section below
//
// $heading = 'Most Recent Topics You Started';
// $where_clause = 'ms.ID_MEMBER = '.$ID_MEMBER;
// $limit_clause = 'LIMIT 5';
// $order_clause = 't.ID_FIRST_MSG DESC';
////////////////////////////////////////////   

//////////////////////////////////////////// ---------- Boardindex Most Recent Topics Arguments
//
// Comment out the Info Center's Most Recent Posts Code and Insert this
// to show Most Recent Topics in full detail style instead
//
// $list_count = $settings['number_recent_posts'];
// $heading = 'Most Recently Posted To Topics';
// $where_clause = 't.ID_LAST_MSG >= ' . ($modSettings['maxMsgID'] - 50 * min($list_count, 5));
// $limit_clause = 'LIMIT ' . $list_count;
// $order_clause = 't.ID_LAST_MSG DESC';
////////////////////////////////////////////   

////////////////////////////////////////////  ------ Remove down to next mark to use in block/boardindex ------
//
if (empty($settings['number_recent_posts']))
$number_recent_posts = 20;
else
$number_recent_posts = $settings['number_recent_posts'];

        if( isset($_GET['type']) )
$list_type = $_GET['type'];
else
$list_type = 'last';

        if( isset($_GET['count']) )
$list_count = $_GET['count'];
else
{
$list_count = $number_recent_posts;
if ($list_type == 'notify')
$list_count = 100;
elseif ($list_type == 'hours')
$list_count = 24;
}

if ($list_count <= 0)
{
$list_count = $number_recent_posts;
if ($list_type == 'hours')
$list_count = 24;
}

if ($list_count > 100)
$list_count = 100;

        if( isset($_GET['order']) )
$list_order = $_GET['order'];
else
$list_order = 'lastpost';

        if( $list_order == 'create' )
$order_clause = 't.ID_FIRST_MSG DESC';
else
$order_clause = 't.ID_LAST_MSG DESC';

if ($list_type == 'hours')
{
$list_from = strtotime($list_count.' hours ago');
$where_clause = 'ml.posterTime >= ' . $list_from;
$limit_clause = ' ';
$heading = 'Topics Posted To In Last '. $list_count . ' Hours';
}
elseif ($list_type == 'unreplied')
{
$where_clause = 't.numReplies = 0';
if ($list_count == 0)
{
$limit_clause = ' ';
$heading = 'Unreplied To Topics';
}
else
{
$limit_clause = 'LIMIT ' . $list_count;
$heading = $list_count . ' Most Recent Unreplied To Topics';
}
}
elseif ($list_type == 'notify')
{
if (isset($_GET['user']) && $user_info['is_admin'])
{
$announce_topics = array();
$request = db_query("SELECT ID_TOPIC FROM {$db_prefix}log_notify WHERE ID_MEMBER = {$_GET['user']} AND ID_BOARD = 0", __FILE__, __LINE__);
while ($row = mysql_fetch_assoc($request))
$watched_topics[] = $row['ID_TOPIC'];
mysql_free_result($request);
$heading = $list_count . ' Most Recent Topics Being Watched by User # '.$_GET['user'];
$where_clause = 't.ID_TOPIC IN (' . implode(', ', $watched_topics) . ')';
$limit_clause = 'LIMIT ' . $list_count;
if (empty($watched_topics))
$do_query = 0;
}
else
{
$announce_topics = array();
$request = db_query("SELECT ID_TOPIC FROM {$db_prefix}log_notify WHERE ID_MEMBER = {$ID_MEMBER} AND ID_BOARD = 0", __FILE__, __LINE__);
while ($row = mysql_fetch_assoc($request))
$watched_topics[] = $row['ID_TOPIC'];
mysql_free_result($request);
$heading = $list_count . ' Most Recent Topics Being Watched';
$where_clause = 't.ID_TOPIC IN (' . implode(', ', $watched_topics) . ')';
$limit_clause = 'LIMIT ' . $list_count;
if (empty($watched_topics))
$do_query = 0;
}
}
else
{
$where_clause = 't.ID_LAST_MSG >= ' . ($modSettings['maxMsgID'] - 90 * min($list_count, 5));
$limit_clause = 'LIMIT ' . $list_count;
$heading = $list_count . ' Most Recently Posted To Topics';
}
//
////////////////////////////////////////////  ------ Remove up to first mark to use in block/boardindex ------

$stable_icons = array('xx', 'thumbup', 'thumbdown', 'exclamation', 'question', 'lamp', 'smiley', 'angry', 'cheesy', 'grin', 'sad', 'wink', 'moved', 'recycled', 'wireless');
$icon_sources = array();
foreach ($stable_icons as $icon)
$icon_sources[$icon] = 'images_url';

$topics = array();

if ($do_query == 1)
{
$request = db_query("
SELECT
ms.subject AS firstSubject, ms.posterTime AS firstPosterTime, ms.ID_TOPIC, t.ID_BOARD, b.name AS bname,
t.numReplies, t.numViews, ms.ID_MEMBER AS ID_FIRST_MEMBER, ml.ID_MEMBER AS ID_LAST_MEMBER,
ml.posterTime AS lastPosterTime, IFNULL(mems.realName, ms.posterName) AS firstPosterName,
IFNULL(meml.realName, ml.posterName) AS lastPosterName, ml.subject AS lastSubject, b.memberGroups,
ml.icon AS lastIcon, ms.icon AS firstIcon, t.ID_POLL, t.isSticky, t.locked, ml.modifiedTime AS lastModifiedTime,
LEFT(ml.body, 384) AS lastBody, LEFT(ms.body, 384) AS firstBody,
ml.smileysEnabled AS lastSmileys, ms.smileysEnabled AS firstSmileys, t.ID_FIRST_MSG, t.ID_LAST_MSG,"
. ($user_info['is_guest'] ? '1 AS isRead, 0 AS new_from' : '
IFNULL(lt.ID_MSG, IFNULL(lmr.ID_MSG, 0)) >= ml.ID_MSG_MODIFIED AS isRead,
IFNULL(lt.ID_MSG, IFNULL(lmr.ID_MSG, -1)) + 1 AS new_from') . "
FROM ({$db_prefix}messages AS ms, {$db_prefix}messages AS ml, {$db_prefix}topics AS t, {$db_prefix}boards AS b)
LEFT JOIN {$db_prefix}members AS mems ON (mems.ID_MEMBER = ms.ID_MEMBER)
LEFT JOIN {$db_prefix}members AS meml ON (meml.ID_MEMBER = ml.ID_MEMBER)
LEFT JOIN {$db_prefix}log_topics AS lt ON (lt.ID_TOPIC = t.ID_TOPIC AND lt.ID_MEMBER = $ID_MEMBER)
LEFT JOIN {$db_prefix}log_mark_read AS lmr ON (lmr.ID_BOARD = t.ID_BOARD AND lmr.ID_MEMBER = $ID_MEMBER)
WHERE " . $where_clause . $ex_board_clause . "
AND t.ID_TOPIC = ms.ID_TOPIC
AND b.ID_BOARD = t.ID_BOARD" . (!empty($modSettings['recycle_enable']) && $modSettings['recycle_board'] > 0 ? " AND b.ID_BOARD != $modSettings[recycle_board]" : '') . "
AND ms.ID_MSG = t.ID_FIRST_MSG
AND ml.ID_MSG = t.ID_LAST_MSG
AND " . $user_info['query_see_board'] . "
ORDER BY " . $order_clause . " " . $limit_clause, __FILE__, __LINE__);


$topic_ids = array();
while ($row = mysql_fetch_assoc($request))
{
if ($row['ID_POLL'] > 0 && $modSettings['pollMode'] == '0')
continue;

$topic_ids[] = $row['ID_TOPIC'];

// Clip the strings first because censoring is slow :/. (for some reason?)
$row['firstBody'] = strip_tags(strtr(parse_bbc($row['firstBody'], $row['firstSmileys'], $row['ID_FIRST_MSG']), array('<br />' => '
')));
if (strlen($row['firstBody']) > 128)
$row['firstBody'] = substr($row['firstBody'], 0, 128) . '...';
$row['lastBody'] = strip_tags(strtr(parse_bbc($row['lastBody'], $row['lastSmileys'], $row['ID_LAST_MSG']), array('<br />' => '
')));
if (strlen($row['lastBody']) > 128)
$row['lastBody'] = substr($row['lastBody'], 0, 128) . '...';

$row['lastSubject'] = $row['firstSubject'];
$row['lastBody'] = $row['firstBody'];



// We need to check the topic icons exist... you can never be too sure!
if (empty($modSettings['messageIconChecks_disable']))
{
// First icon first... as you'd expect.
if (!isset($icon_sources[$row['firstIcon']]))
$icon_sources[$row['firstIcon']] = file_exists($settings['theme_dir'] . '/images/post/' . $row['firstIcon'] . '.gif') ? 'images_url' : 'default_images_url';
// Last icon... last... duh.
if (!isset($icon_sources[$row['lastIcon']]))
$icon_sources[$row['lastIcon']] = file_exists($settings['theme_dir'] . '/images/post/' . $row['lastIcon'] . '.gif') ? 'images_url' : 'default_images_url';
}

        $dummy = array();
        $dummy[] = $row['ID_FIRST_MEMBER'];
        $dummy[] = $row['ID_LAST_MEMBER'];
        loadMemberData($dummy);
        $profile_first = &$user_profile[$row['ID_FIRST_MEMBER']];
        $profile_last = &$user_profile[$row['ID_LAST_MEMBER']];

// And build the array.
$topics[$row['ID_TOPIC']] = array(
'id' => $row['ID_TOPIC'],
'first_post' => array(
'id' => $row['ID_FIRST_MSG'],
'member' => array(
'name' => $row['firstPosterName'],
'id' => $row['ID_FIRST_MEMBER'],
'href' => $scripturl . '?action=profile;u=' . $row['ID_FIRST_MEMBER'],
'link' => !empty($row['ID_FIRST_MEMBER']) ? '<a href="' . $scripturl . '?action=profile;u=' . $row['ID_FIRST_MEMBER'] . '" title="' . $txt[92] . ' ' . $row['firstPosterName'] . '">' . $row['firstPosterName'] . '</a>' : $row['firstPosterName']
),
'time' => timeformat($row['firstPosterTime']),
'timestamp' => forum_time(true, $row['firstPosterTime']),
'subject' => $row['firstSubject'],
'preview' => $row['firstBody'],
'icon' => $row['firstIcon'],
'icon_url' => $settings[$icon_sources[$row['firstIcon']]] . '/post/' . $row['firstIcon'] . '.gif',
'href' => $scripturl . '?topic=' . $row['ID_TOPIC'] . '.0;topicseen',
'link' => '<a href="' . $scripturl . '?topic=' . $row['ID_TOPIC'] . '.0;topicseen">' . $row['firstSubject'] . '</a>'
),
'last_post' => array(
'id' => $row['ID_LAST_MSG'],
'member' => array(
'name' => $row['lastPosterName'],
'id' => $row['ID_LAST_MEMBER'],
'href' => $scripturl . '?action=profile;u=' . $row['ID_LAST_MEMBER'],
'link' => !empty($row['ID_LAST_MEMBER']) ? '<a href="' . $scripturl . '?action=profile;u=' . $row['ID_LAST_MEMBER'] . '">' . $row['lastPosterName'] . '</a>' : $row['lastPosterName']
),
'time' => timeformat($row['lastPosterTime']),
'timestamp' => forum_time(true, $row['lastPosterTime']),
'subject' => $row['lastSubject'],
'preview' => $row['lastBody'],
'icon' => $row['lastIcon'],
'icon_url' => $settings[$icon_sources[$row['lastIcon']]] . '/post/' . $row['lastIcon'] . '.gif',
'href' => $scripturl . '?topic=' . $row['ID_TOPIC'] . ($row['numReplies'] == 0 ? '.0' : '.msg' . $row['ID_LAST_MSG']) . ';topicseen#msg' . $row['ID_LAST_MSG'],
'link' => '<a href="' . $scripturl . '?topic=' . $row['ID_TOPIC'] . ($row['numReplies'] == 0 ? '.0' : '.msg' . $row['ID_LAST_MSG']) . ';topicseen#msg' . $row['ID_LAST_MSG'] . '">' . $row['lastSubject'] . '</a>'
),
'new' => $row['isRead'],
'new_from' => $row['new_from'],
'new_href' => $scripturl . '?topic=' . $row['ID_TOPIC'] . '.msg' . $row['new_from'] . ';topicseen#new',
'href' => $scripturl . '?topic=' . $row['ID_TOPIC'] . ($row['numReplies'] == 0 ? '.0' : '.msg' . $row['new_from']) . ';topicseen' . ($row['numReplies'] == 0 ? '' : 'new'),
'link' => '<a href="' . $scripturl . '?topic=' . $row['ID_TOPIC'] . ($row['numReplies'] == 0 ? '.0' : '.msg' . $row['new_from']) . ';topicseen#msg' . $row['new_from'] . '">' . $row['firstSubject'] . '</a>',
'is_sticky' => !empty($modSettings['enableStickyTopics']) && !empty($row['isSticky']),
'is_locked' => !empty($row['locked']),
'is_poll' => $modSettings['pollMode'] == '1' && $row['ID_POLL'] > 0,
'is_hot' => $row['numReplies'] >= $modSettings['hotTopicPosts'],
'is_very_hot' => $row['numReplies'] >= $modSettings['hotTopicVeryPosts'],
'is_posted_in' => false,
'icon' => $row['firstIcon'],
'icon_url' => $settings[$icon_sources[$row['firstIcon']]] . '/post/' . $row['firstIcon'] . '.gif',
'subject' => $row['firstSubject'],
'pages' => $pages,
'replies' => $row['numReplies'],
'views' => $row['numViews'],
'board' => array(
'id' => $row['ID_BOARD'],
'name' => $row['bname'],
'href' => $scripturl . '?board=' . $row['ID_BOARD'] . '.0',
'link' => '<a href="' . $scripturl . '?board=' . $row['ID_BOARD'] . '.0">' . $row['bname'] . '</a>'
)
);

determineTopicClass($topics[$row['ID_TOPIC']]);
}
mysql_free_result($request);

if (!empty($modSettings['enableParticipation']) && !empty($topic_ids))
{
$result = db_query("
SELECT ID_TOPIC
FROM {$db_prefix}messages
WHERE ID_TOPIC IN (" . implode(', ', $topic_ids) . ")
AND ID_MEMBER = $ID_MEMBER", __FILE__, __LINE__);
while ($row = mysql_fetch_assoc($result))
{
if (empty($topics[$row['ID_TOPIC']]['is_posted_in']))
{
$topics[$row['ID_TOPIC']]['is_posted_in'] = true;
$topics[$row['ID_TOPIC']]['class'] = 'my_' . $topics[$row['ID_TOPIC']]['class'];
}
}
mysql_free_result($result);
}

}

if (!empty($topics))
    {
    echo '
        <div class="tborder" ', $context['browser']['needs_size_fix'] && !$context['browser']['is_ie6'] ? 'style="width: 100%;margin:0px;"' : 'style="margin:0px;"', '>
            <table border="0" width="100%" cellspacing="1" cellpadding="5" class="bordercolor">
   
<tr class="catbg">
<td width="10%" colspan="2"> </td>
<td>', $txt[70], '</td>
<td width="18%" align="center">', $txt[111], '</td>
<td width="2%" align="center">Resp.</td>
<td width="2%" align="center">', $txt[301], '</td>

</tr>';

foreach ($topics as $topic)
{
// Do we want to seperate the sticky and lock status out?
if (!empty($settings['seperate_sticky_lock']) && strpos($topic['class'], 'sticky') !== false)
$topic['class'] = substr($topic['class'], 0, strrpos($topic['class'], '_sticky'));
if (!empty($settings['seperate_sticky_lock']) && strpos($topic['class'], 'locked') !== false)
$topic['class'] = substr($topic['class'], 0, strrpos($topic['class'], '_locked'));

echo '
<tr>
<td class="windowbg2" valign="middle" align="center" width="4%">
<img src="' . $settings['images_url'] . '/topic/' . $topic['class'] . '.gif" alt="" />
</td><td class="windowbg" valign="middle" align="center" width="3%">
<img src="' . $topic['last_post']['icon_url'] . '" alt="" align="middle" />
</td><td class="windowbg2' , $topic['is_sticky'] && !empty($settings['seperate_sticky_lock']) ? '' : '' , '" width="48%" valign="middle">' , $topic['is_locked'] && !empty($settings['seperate_sticky_lock']) ? '
<img src="' . $settings['images_url'] . '/icons/quick_lock.gif" align="right" alt="" style="margin: 0;" />' : '' , $topic['is_sticky'] && !empty($settings['seperate_sticky_lock']) ? '
<img src="' . $settings['images_url'] . '/icons/show_sticky.gif" align="right" alt="" style="margin: 0;" />' : '';
if ($topic['new'] == 0)
{
echo '<a href="', $topic['new_href'], '"><img src="', $settings['images_url'], '/', $context['user']['language'], '/new.gif" alt="', $txt[302], '" border="0"/></a> ';
echo '<b>',  $topic['last_post']['link'], '</b>';

}
else
{
echo $topic['first_post']['link'];
}
echo '

<td class="windowbg" valign="middle" width="18%">
<div align=right><span style="font-size:8.5pt">
', $topic['last_post']['time'], '<br />
<a href="', $topic['last_post']['href'], '"><img src="', $settings['images_url'], '/icons/last_post.gif" alt="', $txt[111], '" title="', $txt[111], '" style="float: right;" /></a>
', $txt[525], ' ', $topic['last_post']['member']['link'], '   
</span></div>
</td>
<td class="windowbg2" valign="middle" width="2%" align="center">
', $topic['replies'], '</td>
<td class="windowbg" valign="middle" width="2%" align="center">
', $topic['views'], '</td>


</tr>';
}

    echo '</table></div>';

    }
else
    echo '<b><u>'.$heading.'<br><br>No Topics Match Search Criteria</u></b>';

Thanks a lot!!

[jacortina]

Well, that's NOT the latest version of the code (not the most recent of the most recent blocks). In fact one of the changes made was to cut down on queries.

See if the code in this message helps:
http://www.tinyportal.net/index.php?topic=596.msg105580#msg105580

And 5 queries for a block like the Calendar block isn't that bad. TP is going to execute a query just to load the php to parse and then whatever queries are done in the block (some of which are on the SMF end because the block calls built-in functions.

There IS some optimization possible and it looks like I have more work to do there.

[dannbass]

Fantastic!!! Thank you very much!! Now it went down to 42 queries!

Thanks a lot again!!

[G6Cad]

Thank you for the help on this J.A.Cortina